IP Library › Granted Patent US 12,737,969
Granted Patent B2
US 12,737,969 · App. 18/752,598 · Granted Sep 15, 2026

Interactive image generation

Inventors: Clarence Chui (Los Altos Hills, CA); Manu Parmar (Sunnyvale, CA); Marc Mignard (Los Gatos, CA); Avinash Venkata Uppuluri (Sunnyvale, CA); John Le (Santa Clara, CA); Andrew Pang (Fremont, CA)
Assignee: Outward, Inc.
G06T15/506G06T7/10G06T7/50G06T7/70G06T15/04G06T15/50G06T15/60G06T17/20G06T19/006G06T19/20G06V10/70G06V20/20G06V20/36H04N5/222H04N23/56H04N23/74G06T2200/08G06T2200/24G06T2207/10028G06T2207/20081G06T2207/20084G06T2207/20212G06T2207/30244G06T2210/04G06T2210/56G06T2219/2004G06T2219/2016H04N13/239
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,737,969
App. No.
18/752,598
Granted
Sep 15, 2026
Kind
B2
Abstract

A content generation platform is generally described herein. More specifically, interactive image generation and techniques and features thereof are disclosed herein. One or more sets of images of a scene are captured in an imaging studio. The captured one or more sets of images of the scene are processed using one or more machine learning based networks to generate an interactive image of the scene comprising a plurality of interactive features. One or more of the plurality of interactive features of the generated interactive image may be modified or edited according to user preferences.

Claims (62)

1 . A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:

activating one or more sets of components of an imaging studio to capture a set of images of a scene comprising the imaging studio; and

providing the set of images of the scene captured in the imaging studio to an artificial intelligence based processing stage to generate an interactive image of the scene, wherein the generated interactive image of the scene comprises a two-dimensional image with at least partial three-dimensional capabilities but without having an underlying three-dimensional model and wherein the at least partial three-dimensional capabilities of the generated interactive image of the scene are facilitated by one or more machine learning based networks of the artificial intelligence based processing stage that have at least in part been trained on training images generated from corresponding three-dimensional models.

2 . The computer program product of claim 1 , further comprising computer instructions for guiding placement of one or more objects comprising the scene in the imaging studio.

3 . The computer program product of claim 2 , wherein placement of the one or more objects in the imaging studio is guided by one or more corresponding object placement templates.

4 . The computer program product of claim 2 , wherein guiding placement comprises providing guidelines or markers indicating placement.

5 . The computer program product of claim 1 , further comprising computer instructions for adjusting one or more components comprising the imaging studio.

6 . The computer program product of claim 5 , wherein adjusting one or more components comprising the imaging studio comprises automatically adjusting a lighting array comprising the imaging studio or a part thereof.

7 . The computer program product of claim 5 , wherein adjusting one or more components comprising the imaging studio comprises automatically adjusting a camera comprising the imaging studio.

8 . The computer program product of claim 5 , wherein adjusting one or more components comprising the imaging studio comprises automatically adjusting a turntable comprising the imaging studio.

9 . The computer program product of claim 1 , wherein activating one or more sets of components comprises selectively activating one or more lighting panels and one or more cameras to capture the set of images of the scene.

10 . The computer program product of claim 1 , wherein the set of images of the scene comprises a prescribed view of the scene.

11 . The computer program product of claim 1 , wherein the set of images of the scene comprises multiple and diverse views of the scene.

12 . The computer program product of claim 1 , wherein the set of images of the scene comprises images captured with different lighting conditions.

13 . The computer program product of claim 1 , wherein the set of images of the scene comprises images captured with different camera poses.

14 . The computer program product of claim 1 , wherein the computer program product operates as a controller of the imaging studio and components thereof.

15 . The computer program product of claim 1 , wherein components of the imaging studio are adjusted and activated via control signals from the computer program product.

16 . The computer program product of claim 1 , wherein the computer program product runs on a computing device situated outside the imaging studio.

17 . The computer program product of claim 1 , wherein a configurable option associated with the capture is automatically selected or user selected via a user interface.

18 . The computer program product of claim 1 , wherein providing the captured set of images of the scene to the artificial intelligence based processing stage comprises uploading the captured set of images of the scene to a cloud application.

19 . A system, comprising:

a processor configured to:

activate one or more sets of components of an imaging studio to capture a set of images of a scene comprising the imaging studio; and

provide the set of images of the scene captured in the imaging studio to an artificial intelligence based processing stage to generate an interactive image of the scene, wherein the generated interactive image of the scene comprises a two-dimensional image with at least partial three-dimensional capabilities but without having an underlying three-dimensional model and wherein the at least partial three-dimensional capabilities of the generated interactive image of the scene are facilitated by one or more machine learning based networks of the artificial intelligence based processing stage that have at least in part been trained on training images generated from corresponding three-dimensional models; and

a memory coupled to the processor and configured to provide the processor with instructions.

20 . A method, comprising:

activating one or more sets of components of an imaging studio to capture a set of images of a scene comprising the imaging studio; and

providing the set of images of the scene captured in the imaging studio to an artificial intelligence based processing stage to generate an interactive image of the scene, wherein the generated interactive image of the scene comprises a two-dimensional image with at least partial three-dimensional capabilities but without having an underlying three-dimensional model and wherein the at least partial three-dimensional capabilities of the generated interactive image of the scene are facilitated by one or more machine learning based networks of the artificial intelligence based processing stage that have at least in part been trained on training images generated from corresponding three-dimensional models.

21 . The system of claim 19 , wherein the processor is further configured to guide placement of one or more objects comprising the scene in the imaging studio.

22 . The system of claim 21 , wherein placement of the one or more objects in the imaging studio is guided by one or more corresponding object placement templates.

23 . The system of claim 21 , wherein to guide placement comprises to provide guidelines or markers indicating placement.

24 . The system of claim 19 , wherein the processor is further configured to adjust one or more components comprising the imaging studio.

25 . The system of claim 24 , wherein to adjust one or more components comprising the imaging studio comprises to automatically adjust a lighting array comprising the imaging studio or a part thereof.

26 . The system of claim 24 , wherein to adjust one or more components comprising the imaging studio comprises to automatically adjust a camera comprising the imaging studio.

27 . The system of claim 24 , wherein to adjust one or more components comprising the imaging studio comprises to automatically adjust a turntable comprising the imaging studio.

28 . The system of claim 19 , wherein to activate one or more sets of components comprises to selectively activate one or more lighting panels and one or more cameras to capture the set of images of the scene.

29 . The system of claim 19 , wherein the set of images of the scene comprises a prescribed view of the scene.

30 . The system of claim 19 , wherein the set of images of the scene comprises multiple and diverse views of the scene.

31 . The system of claim 19 , wherein the set of images of the scene comprises images captured with different lighting conditions.

32 . The system of claim 19 , wherein the set of images of the scene comprises images captured with different camera poses.

33 . The system of claim 19 , wherein the system operates as a controller of the imaging studio and components thereof.

34 . The system of claim 19 , wherein components of the imaging studio are adjusted and activated via control signals from the system.

35 . The system of claim 19 , wherein the system is situated outside the imaging studio.

36 . The system of claim 19 , wherein a configurable option associated with the capture is automatically selected or user selected via a user interface.

37 . The system of claim 19 , wherein to provide the captured set of images of the scene to the artificial intelligence based processing stage comprises to upload the captured set of images of the scene to a cloud application.

38 . The method of claim 20 , further comprising guiding placement of one or more objects comprising the scene in the imaging studio.

39 . The method of claim 38 , wherein placement of the one or more objects in the imaging studio is guided by one or more corresponding object placement templates.

40 . The method of claim 38 , wherein guiding placement comprises providing guidelines or markers indicating placement.

41 . The method of claim 20 , further comprising adjusting one or more components comprising the imaging studio.

42 . The method of claim 41 , wherein adjusting one or more components comprising the imaging studio comprises automatically adjusting a lighting array comprising the imaging studio or a part thereof.

43 . The method of claim 41 , wherein adjusting one or more components comprising the imaging studio comprises automatically adjusting a camera comprising the imaging studio.

44 . The method of claim 41 , wherein adjusting one or more components comprising the imaging studio comprises automatically adjusting a turntable comprising the imaging studio.

45 . The method of claim 20 , wherein activating one or more sets of components comprises selectively activating one or more lighting panels and one or more cameras to capture the set of images of the scene.

46 . The method of claim 20 , wherein the set of images of the scene comprises a prescribed view of the scene.

47 . The method of claim 20 , wherein the set of images of the scene comprises multiple and diverse views of the scene.

48 . The method of claim 20 , wherein the set of images of the scene comprises images captured with different lighting conditions.

49 . The method of claim 20 , wherein the set of images of the scene comprises images captured with different camera poses.

50 . The method of claim 20 , wherein the method is used by a controller of the imaging studio and components thereof.

51 . The method of claim 20 , wherein components of the imaging studio are adjusted and activated via control signals from a controller.

52 . The method of claim 20 , wherein the method is used by a controller situated outside the imaging studio.

53 . The method of claim 20 , wherein a configurable option associated with the capture is automatically selected or user selected via a user interface.

54 . The method of claim 20 , wherein providing the captured set of images of the scene to the artificial intelligence based processing stage comprises uploading the captured set of images of the scene to a cloud application.

Continuity (3)
Continuation 17966246 · Oct 14, 2022
Provisional Application 63255886 · Oct 14, 2021
Related Publication 20240346748A1 · Oct 17, 2024
References Cited (112)
US 6034739A · Rohlfing · 2000 [cited by applicant]
US 7420555B1 · Lee · 2008 [cited by applicant]
US 8010906B1 · Sinnard · 2011 [cited by applicant]
US 8301022B1 · McGuire · 2012 [cited by examiner]
US 9495796B2 · Breton · 2016 [cited by applicant]
US 9536340B2 · Loberg · 2017 [cited by applicant]
US 9552674B1 · Jayadevaprakash · 2017 [cited by applicant]
US 9619931B1 · Klabunde · 2017 [cited by applicant]
US 9990760B2 · Aguilera Perez · 2018 [cited by applicant]
US 10109107B2 · Knorr · 2018 [cited by applicant]
US 10127727B1 · Yuan · 2018 [cited by applicant]
US 10168789B1 · Soto · 2019 [cited by applicant]
US 10210664B1 · Chaturvedi · 2019 [cited by applicant]
US 10373380B2 · Kutliroff · 2019 [cited by applicant]
US 10692277B1 · Sunkavalli · 2020 [cited by applicant]
US 10699488B1 · Terrano · 2020 [cited by applicant]
US 10885701B1 · Patel · 2021 [cited by applicant]
US 11200718B1 · Meyer · 2021 [cited by applicant]
US 11375179B1 · Clemens · 2022 [cited by applicant]
US 11496691B2 · Chapman · 2022 [cited by applicant]
US 11666825B2 · Delamont · 2023 [cited by applicant]
US 11856674B1 · Sanchez · 2023 [cited by applicant]
US 11864285B2 · Beaumier · 2024 [cited by applicant]
US 11935199B2 · Faaborg · 2024 [cited by applicant]
US 11983853B1 · Zhu · 2024 [cited by examiner]
US 20010032062A1 · Plaskoff · 2001 [cited by applicant]
US 20030038875A1 · Ivanov · 2003 [cited by applicant]
US 20030174286A1 · Trumbull · 2003 [cited by applicant]
US 20040095474A1 · Matsufune · 2004 [cited by applicant]
US 20050104966A1 · Schoelkopf · 2005 [cited by applicant]
US 20070057815A1 · Foy · 2007 [cited by applicant]
US 20070133880A1 · Sun · 2007 [cited by applicant]
US 20100158379A1 · Hatfield · 2010 [cited by applicant]
US 20110242334A1 · Wilburn · 2011 [cited by applicant]
US 20120002871A1 · Hu · 2012 [cited by applicant]
US 20130218714A1 · Watkins · 2013 [cited by applicant]
US 20130272609A1 · Sun · 2013 [cited by applicant]
US 20140129990A1 · Xin · 2014 [cited by applicant]
US 20140368530A1 · James · 2014 [cited by applicant]
US 20150062125A1 · Aguilera Perez · 2015 [cited by applicant]
US 20150130790A1 · Vasquez, II · 2015 [cited by applicant]
US 20150250041A1 · Jobe · 2015 [cited by applicant]
US 20150302116A1 · Howell · 2015 [cited by applicant]
US 20160180419A1 · Adeyoola · 2016 [cited by applicant]
US 20160180441A1 · Hasan · 2016 [cited by examiner]
US 20160260260A1 · Fei · 2016 [cited by applicant]
US 20170064214A1 · Zhang · 2017 [cited by applicant]
US 20170103584A1 · Vats · 2017 [cited by applicant]
US 20180189974A1 · Clark · 2018 [cited by applicant]
US 20180255290A1 · Holzer · 2018 [cited by applicant]
US 20190043243A1 · Chui · 2019 [cited by applicant]
US 20190102949A1 · Sheftel · 2019 [cited by applicant]
US 20190121522A1 · Davis · 2019 [cited by applicant]
US 20190227172A1 · Schaefer · 2019 [cited by applicant]
US 20190227419A1 · Mcnelley · 2019 [cited by applicant]
US 20190318542A1 · Sai Krishna · 2019 [cited by applicant]
US 20190332866A1 · Beall · 2019 [cited by applicant]
US 20200005448A1 · Subramanian · 2020 [cited by applicant]
US 20200098122A1 · Dal Mutto · 2020 [cited by applicant]
US 20200134383A1 · Rhee · 2020 [cited by applicant]
US 20200175759A1 · Russell · 2020 [cited by applicant]
US 20200250879A1 · Foster · 2020 [cited by applicant]
US 20200279364A1 · Sarkisian · 2020 [cited by applicant]
US 20200302681A1 · Totty · 2020 [cited by applicant]
US 20200302686A1 · Totty · 2020 [cited by applicant]
US 20200312042A1 · Sardari · 2020 [cited by applicant]
US 20200342652A1 · Rowell · 2020 [cited by applicant]
US 20200394849A1 · Barker · 2020 [cited by applicant]
US 20200410725A1 · Haines · 2020 [cited by applicant]
US 20210012561A1 · Sunkavalli · 2021 [cited by applicant]
US 20210042950A1 · Wantland · 2021 [cited by applicant]
US 20210049811A1 · Fedyukov · 2021 [cited by applicant]
US 20210142497A1 · Pugh · 2021 [cited by applicant]
US 20210174604A1 · Long · 2021 [cited by applicant]
US 20210183080A1 · Hoiem · 2021 [cited by applicant]
US 20210201526A1 · Moloney · 2021 [cited by applicant]
US 20210224889A1 · Colby · 2021 [cited by applicant]
US 20210241478A1 · Putman · 2021 [cited by applicant]
US 20210279957A1 · Eder · 2021 [cited by applicant]
US 20210327119A1 · Malin · 2021 [cited by applicant]
US 20210350634A1 · Major · 2021 [cited by applicant]
US 20210383115A1 · Alon · 2021 [cited by applicant]
US 20210407169A1 · Swann · 2021 [cited by applicant]
US 20220005099A1 · Mordovskoi · 2022 [cited by applicant]
US 20220028159A1 · Vincent · 2022 [cited by applicant]
US 20220084296A1 · Sadalgi · 2022 [cited by applicant]
US 20220101507A1 · Tournier · 2022 [cited by applicant]
US 20220101556A1 · Zhang · 2022 [cited by examiner]
US 20220114291A1 · Li · 2022 [cited by applicant]
US 20220165023A1 · Lino · 2022 [cited by applicant]
US 20220269888A1 · Stoeva · 2022 [cited by applicant]
US 20220284551A1 · Neofytou · 2022 [cited by applicant]
US 20220327886A1 · Mathur · 2022 [cited by applicant]
US 20220343591A1 · Jutan · 2022 [cited by applicant]
US 20220366263A1 · Ji · 2022 [cited by applicant]
US 20230022108A1 · Ghosh · 2023 [cited by applicant]
US 20230079580A1 · Russell · 2023 [cited by applicant]
US 20230085518A1 · Chen · 2023 [cited by applicant]
US 20230093087A1 · Babinowich · 2023 [cited by applicant]
US 20230102949A1 · Boyle · 2023 [cited by applicant]
US 20230104172A1 · Kikuchi · 2023 [cited by applicant]
US 20230134130A1 · Robert · 2023 [cited by applicant]
US 20230154395A1 · Mcnelley · 2023 [cited by applicant]
US 20230260236A1 · Rump · 2023 [cited by applicant]
US 20240037769A1 · Atwood · 2024 [cited by applicant]
US 20240090795A1 · Heng · 2024 [cited by applicant]
Nguyen et al., A Robust 3D-2D Interactive Tool for Scene Segmentation and Annotation, IEEE Transactions on Visualization and Computer Graphics, vol. 24, No. 12, Dec. 2018, pp. 3005-3018. [cited by applicant]
Chen et al., Deep Learning Approach to Point Cloud Scene Understanding for Automated Scan to 3D Reconstruction, Journal of Computing in Civil Engineering, vol. 33, Issue 4, May 2019, 19 pages. [cited by applicant]
Havran et al., Interactive System for Dynamic Scene Lighting using Captured Video Environment Maps, Eurographics Symposium on Rendering, 2005, 14 pages. [cited by applicant]
Zou et al., LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 2051-2059. [cited by applicant]
Sadalgi et al., U.S. Appl. No. 63/079,429 of PG Pub No. 2022/0084296 A1, filed Sep. 16, 2020. [cited by applicant]
Yang et al., “3D Object Relighting Based on Multi-View Stereo and Image Based Lighting Techniques” 2009 IEEE International Conference on Multimedia and Expo, IEEE, 2009. [cited by applicant]