IP Library Granted Patent US 12,367,232
Granted Patent B2
US 12,367,232 · App. 17/606,862 · Granted Jul 22, 2025

Generative image acquisition

Inventors: Dehua Cui (Redmond, WA); Albert Joseph Kishan Thambiratnam (Redmond, CN); Lin Su (Redmond, WA)
Assignee: Microsoft Technology Licensing, LLC
G06F16/532G06F16/538
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,367,232
App. No.
17/606,862
Granted
Jul 22, 2025
Kind
B2
Abstract

The present disclosure provides method and apparatus for generative image acquisition. A query can be received. A first set of images retrieved according to the query can be obtained. It is can be determined that the first set of images includes a first image that partially satisfies the query. A missing component of the first image compared to the query can be detected. A second set of images based on the first image and the missing component can be generated.

Claims (36)

1. A method for generative image acquisition, comprising:

receiving a query that specifies a first component and a second component relating to features of an image;

obtaining a first set of images retrieved according to the query;

determining that the first set of images has the first component and a third component, the third component being different from the second component, wherein the second component has an attribute and the third component has an attribute;

comparing the first set of images to the query;

determining that the second component is missing from the first set of images compared to the query thereby identifying the second component as a missing component;

modifying one of the first set of images to replace the third component with the second component to generate a second set of images based on the first set of images, the second set of images including the second component, wherein a generating module modifies the one of the first set of images by changing the third component attribute in the one of the first set of images to the second component attribute; and

presenting the second set of images.

2. The method of claim 1 , wherein each one of the first component, the second component, and the third component includes an object and/or an attribute associated with the object.

3. The method of claim 1 , further comprising:

constructing a derived query based on the missing component;

obtaining a third set of images retrieved according to the derived query;

combining the first set of images and the third set of images; and

adding the combined first and third sets of images into the second set of images.

4. The method of claim 3 , wherein a matching degree of an additional image with the derived query satisfies a matching degree threshold.

5. The method of claim 1 , further comprising:

determining that a number of images included in the first set of images and the second set of images that satisfies the query meets a predetermined threshold; and

stopping the modifying.

6. An apparatus for generative image acquisition, comprising:

a receiving module for receiving a query that specifies a first component and a second component relating to features of an image;

an obtaining module for obtaining a first set of images retrieved according to the query;

a determining module for determining that the first set of images has the first component and a third component, the third component being different from the second component, wherein the second component has an attribute and the third component has an attribute;

a module for comparing the first set of images to the query;

a module for determining that the second component is missing from the first set of images compared to the query thereby identifying the second component as a missing component;

a generating module for modifying one of the first set of images to replace the third component with the second component to generate a second set of images based on the first set of images, the second set of images including the second component, wherein a generating module modifies the one of the first set of images by changing the third component attribute in the one of the first set of images to the second component attribute; and

a module for presenting the second set of images.

7. An apparatus for generative image acquisition, comprising:

at least one processor; and

a memory storing computer executable instructions that, when executed, cause the at least one processor to:

receiving a query that specifies a first component and a second component relating to features of an image;

obtaining a first set of images retrieved according to the query;

determining that the first set of images has the first component and a third component, the third component being different from the second component, wherein the second component has an attribute and the third component has an attribute;

comparing the first set of images to the query;

determining that the second component is missing from the first set of images compared to the query thereby identifying the second component as a missing component;

modifying one of the first set of images to replace the third component with the second component to generate a second set of images based on the first set of images, the second set of images including the second component, wherein a generating module modifies the one of the first set of images by changing the third component attribute in the one of the first set of images to the second component attribute; and

presenting the second set of images.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 1, 2021
From: CUI, DEHUA; SU, LIN; THAMBIRATNAM, ALBERT
To: MICROSOFT TECHNOLOGY LICENSING, LLC
Reel/Frame 057980/0839 →
Priority Claims (1)
CN 201910422730.3 · May 21, 2019 · national
Continuity (1)
Related Publication 20220207076A1 · Jun 30, 2022
References Cited (41)
US 7610306B2 · Lin et al. · 2009 [cited by applicant]
US 8762390B2 · Yang et al. · 2014 [cited by applicant]
US 8983941B1 · Murphy-chutorian et al. · 2015 [cited by applicant]
US 9325804B2 · Hamilton et al. · 2016 [cited by applicant]
US 9449026B2 · Wang · 2016 [cited by examiner]
US 10169896B2 · Balasubramanian · 2019 [cited by examiner]
US 10613726B2 · Cohen · 2020 [cited by examiner]
US 11563813B1 · Bhushan · 2023 [cited by examiner]
US 20040078300A1 · Smith · 2004 [cited by examiner]
US 20080040317A1 · Dettinger et al. · 2008 [cited by applicant]
US 20080205756A1 · Kamata · 2008 [cited by examiner]
US 20090287655A1 · Bennett · 2009 [cited by applicant]
US 20090324100A1 · Kletter · 2009 [cited by examiner]
US 20100211602A1 · Menon et al. · 2010 [cited by applicant]
US 20110072048A1 · Hua et al. · 2011 [cited by applicant]
US 20140253701A1 · Wexler · 2014 [cited by examiner]
US 20140280113A1 · Hohwald · 2014 [cited by examiner]
US 20150169643A1 · Kogan et al. · 2015 [cited by applicant]
US 20150363481A1 · Haynes · 2015 [cited by examiner]
US 20160048934A1 · Gross · 2016 [cited by examiner]
US 20160379091A1 · Lin · 2016 [cited by examiner]
US 20170004383A1 · Lin · 2017 [cited by examiner]
US 20180039638A1 · Krivokon · 2018 [cited by examiner]
US 20180068474A1 · Mowatt · 2018 [cited by examiner]
US 20200112705A1 · Yokokawa · 2020 [cited by examiner]
US 20200184278A1 · Zadeh · 2020 [cited by examiner]
CN 101479728A · 2009 [cited by applicant]
CN 104969223A · 2015 [cited by applicant]
CN 105701767A · 2016 [cited by applicant]
CN 107533567A · 2018 [cited by applicant]
Dong, et al., “Semantic Image Synthesis via Adversarial Learning”, in Proceedings of the IEEE International Conference on Computer Vision, Oct. 22, 2017, pp. 5707-5715. [cited by applicant]
Johnson, et al., “Semantic Photo Synthesis”, in the Journal of Eurographics, vol. 25, Issue 3, Sep. 2006, 7 Pages. [cited by applicant]
Nie, et al., “Harvesting Visual Concepts for Image Search with Complex Queries”, in Proceedings of the 20th ACM international conference on Multimedia, Oct. 29, 2012, pp. 59-68. [cited by applicant]
“International Search Report and Written Opinion Issued in PCT Application No. PCT/US20/023049”, Mailed Date: May 27, 2020, 13 Pages. [cited by applicant]
Siddiquie, et al., “Multi-Modal Image Retrieval for Complex Queries using Small Codes”, in Proceedings of International Conference on Multimedia Retrieval, Apr. 1, 2014, 8 Pages. [cited by applicant]
“Office Action Issued in European Patent Application No. 20719256.8”, Mailed Date: Sep. 11, 2023, 7 Pages. [cited by applicant]
Office Action received in Chinese Application No. 201910422730.3, mailed on Feb. 4, 2024, 8 pages (English Translation Provided). [cited by applicant]
Dong, et al., “Semantic Image Synthesis via Adversarial Learning”, Proceedings of the IEEE international conference on computer vision, 2017, pp. 5706-5714. [cited by applicant]
Johnson, et al., “Semantic Photo Synthesis”, Computer Graphics Forum, vol. 25, Issue. 3, Dec. 7, 2006, pp. 407-413. [cited by applicant]
Summons to attend oral proceedings pursuant to Rule 115(1) Received in European Patent Application No. 20719256.8, mailed on Jun. 27, 2024, 08 pages. [cited by applicant]
Notice of Allowance received in Chinese Application No. 201910422730.3, mailed on Jun. 3, 2024, 04 pages (English Translation Provided). [cited by applicant]