IP Library › Granted Patent US 12,190,581
Granted Patent B2
US 12,190,581 · App. 18/244,298 · Granted Jan 7, 2025

Automated usability assessment of buildings using visual data of captured in-room images

Inventors: Viktoriya Stoeva (Phoenix, AZ); Sing Bing Kang (Redmond, WA); Naji Khosravan (Seattle, WA); Lambert E. Wixson (Bellevue, WA)
Assignee: MFTB Holdco, Inc.
G06V20/20G06N3/08G06T7/55G06T19/006G06F16/90G06T2210/04G06T2210/12G06T2210/61
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,190,581
App. No.
18/244,298
Granted
Jan 7, 2025
Kind
B2
Abstract

Techniques are described for automated operations related to analyzing visual data from images captured in rooms of a building and optionally additional captured data about the rooms to assess room layout and other usability information for the building's rooms and optionally for the overall building, and to subsequently using the assessed usability information in one or more further automated manners, such as to improve navigation of the building. The automated operations may include identifying one or more objects in each of the rooms to assess, evaluating one or more target attributes of each object, assessing usability of each object using its target attributes' evaluations and each room using its objects' assessment and other room information with respect to an indicated purpose, and combining the assessments of multiple rooms in a building and other building information to assess usability of the building with respect to its indicated purpose.

Claims (39)

1. A computer-implemented method comprising:

obtaining, by one or more computing systems, one or more panorama images captured in a room of a house and having visual data that in combination provides 360 degrees of horizontal visual data of the room;

determining, by the one or more computing systems, and for each of multiple objects installed in the room, one or more target attributes of that object for which to capture additional data, including analyzing the visual data of the one or more panorama images to identify the multiple objects and to determine locations of the multiple objects in the room and to determine that the visual data of the one or more panorama images lacks details sufficient to satisfy a defined threshold about the one or more target attributes of each of the multiple objects;

providing, by the one or more computing systems, instructions about capturing the additional data about the one or more target attributes of each of the multiple objects and that include information about the determined locations of the multiple objects, wherein the capturing of the additional data includes obtaining additional perspective images of one or more specified types of the multiple objects;

determining, by the one or more computing systems, an assessment of a current usability of the room for an indicated purpose, including assessing current contributions of the multiple objects to the current usability of the room based at least in part on information from the additional data about the one or more target attributes of each of the multiple objects, and including combining information about the assessed current contributions of the multiple objects; and

displaying, by the one or more computing systems, information about the determined assessment of the current usability of the room along with additional visual information about the room.

2. The computer-implemented method of claim 1 further comprising, before the determining of the assessment of the current usability of the room:

analyzing, by the one or more computing systems and via use of at least one trained neural network, additional visual data of the additional perspective images to, for each of the multiple objects, verify whether the additional data about the one or more target attributes of that object has been captured, including to determine for one of the additional perspective images that the one additional perspective image lacks any of the additional data by failing to include visual data that satisfies the defined threshold about any target attribute of any of the multiple objects; and

providing, by the one or more computing systems, further instructions to recapture the one additional perspective image, and obtaining a new copy of the one additional perspective image that includes at least some of the additional data.

3. A computer-implemented method comprising:

obtaining, by one or more computing systems, one or more images captured in a room of a house, and information about multiple objects installed in the room;

determining, by the one or more computing systems, and for each of one or more objects of the multiple objects, one or more target attributes of that object for which to capture additional data, including analyzing visual data of the one or more images to identify that the visual data lacks details sufficient to satisfy a defined threshold about the one or more target attributes of that object;

providing, by the one or more computing systems, instructions for capturing the additional data about the one or more target attributes of each of the one or more objects, wherein the capturing of the additional data includes obtaining additional visual data in additional images of one or more specified types of the one or more objects;

analyzing, by the one or more computing systems and via use of at least one trained neural network, the additional visual data of the additional images to, for each of the one or more objects, verify that the additional data about the one or more target attributes of that object has been captured;

determining, by the one or more computing systems, an assessment of a current usability of the room for an indicated purpose, including assessing contributions of the one or more objects to the current usability of the room based at least in part on information from the additional data about the one or more target attributes of each of the one or more objects, and including combining information about the assessed contributions of the one or more objects; and

providing, by the one or more computing systems, information about the determined assessment of the current usability of the room.

4. The computer-implemented method of claim 3 further comprising analyzing, by the one or more computing systems, the visual data of the one or more images to determine, for each of the one or more objects, a type of the object, and wherein the determining of the one or more target attributes of each of the one or more objects for which to capture additional data is based at least in part on the determined type of that object.

5. The computer-implemented method of claim 3 further comprising:

obtaining, by the one or more computing systems, information about labels identifying the one or more objects; and

identifying, by the one or more computing systems and based on the labels, a plurality of target attributes for the one or more objects that include the one or more target attributes of each of the one or more objects.

6. The computer-implemented method of claim 3 further comprising:

obtaining, by the one or more computing systems, information about at least one of a label for the room or a type of the room; and

determining, by the one or more computing systems and based at least in part on the at least one of the label for the room or the type of the room, the one or more objects.

7. The computer-implemented method of claim 3 wherein the additional data includes information about at least one of a shape of the room or a layout of items in the room, and wherein the determining of the assessment of the current usability of the room for the indicated purpose is based at least in part on the information about the at least one of the shape or layout of the room.

8. The computer-implemented method of claim 3 wherein the additional data includes at least one of a type of the room or a label for the room, and wherein the method further comprises determining, by the one or more computing systems, the indicated purpose of the room based at least in part on the at least one of the type of the room or the label for the room.

9. The computer-implemented method of claim 3 wherein the analyzing of the additional visual data of the additional images further includes determining that the additional visual data lacks visual data that satisfies the defined threshold about at least one target attribute of one of the one or more objects, and wherein the method further comprises providing, by the one or more computing systems, further instructions to capture a further additional image with further additional visual data that satisfies the defined threshold about the at least one target attribute of the one object, and using the further additional visual data as part of the assessing the contribution of the one object to the current usability of the room.

10. The computer-implemented method of claim 3 wherein the analyzing of the additional visual data of the additional images further includes determining that one of the additional images lacks any of the additional data, and wherein the method further comprises providing, by the one or more computing systems, further instructions to capture a further copy of the one additional image that includes at least some of the additional data, and using the further copy of the additional image as part of the assessing of the contributions of the one or more objects to the current usability of the room.

11. The computer-implemented method of claim 3 wherein the analyzing of the visual data of the one or more images further includes determining locations of the one or more objects in the one or more images, wherein the providing of the instructions further includes providing information about the determined locations of the one or more objects, and wherein the one or more objects installed in the room each is at least one of a light fixture, or a plumbing fixture, or a piece of built-in furniture, or a built-in structure inside walls of the room, or an electrical appliance, or a gas-powered appliance, or an oil-powered appliance, or an object to produce a fire by burning fuel, or installed flooring, or an installed wall covering, or an installed window covering, or hardware attached to a door, or hardware attached to a window, or an installed countertop.

12. The computer-implemented method of claim 3 wherein the analyzing of the visual data of the one or more images further identifies one or more additional objects in the room that are each a piece of furniture or a moveable item, wherein the determining of the one or more target attributes and the providing of the instructions and the analyzing of the additional visual data and the assessing of the contributions are each further performed for the one or more additional objects, and wherein the determining of the assessment of the current usability of the room for the indicated purpose is further based in part on assessments of contributions for the one or more additional objects to the current usability of the room.

13. The computer-implemented method of claim 3 wherein the assessing of the contributions of the one or more objects to the current usability of the room for the indicated purpose and the determining of the assessment of the current usability of the room for the indicated purpose are performed by assessing properties of the one or more objects and of the room that include at least one of condition, or quality, or functionality, or effectiveness.

14. The computer-implemented method of claim 3 wherein the one or more images include one or more panorama images that in combination include 360 degrees of horizontal visual coverage of the room, wherein the additional images include one or more perspective images each having less than 180 degrees of horizontal visual coverage of the room, wherein the analyzing of the visual data and the analyzing of the additional visual data are performed without using any depth information from any depth-sensing sensors for distances to surrounding surfaces from locations at which the one or more images and the additional images are captured, wherein the additional visual data further includes visual information in at least one of a video or a three-dimensional model about at least one of the one or more objects and/or at least one target attribute, wherein the method further comprises obtaining, by the one or more computing systems, additional non-visual data that has further additional details about at least one object and/or at least one target attribute and that includes at least one of textual data or audio data, and wherein the determining of the contribution of one or more of the one or more objects is further based in part on analysis of the additional non-visual data and of the at least one of the video or the three-dimensional model.

15. A non-transitory computer-readable medium having stored contents that cause one or more computing devices to perform automated operations that include at least:

obtaining, by the one or more computing devices, one or more images captured in a room of a building;

providing, by the one or more computing devices and based at least in part on analysis of visual data of the one or more images, instructions for capturing additional images having additional visual data for multiple objects in the room to provide additional data about one or more indicated target attributes of each of the multiple objects;

analyzing, by the one or more computing devices, the additional visual data of the additional images to, for each of the multiple objects, verify that the additional data about the one or more target attributes of that object has been captured, including to determine that the additional visual data satisfies a defined threshold about at least one target attribute of at least one of the multiple objects;

determining, by the one or more computing devices and using the additional data from the additional images, assessments of the multiple objects that include, for each of the multiple objects, an assessment of a contribution of that object to usability of the room for an indicated purpose based at least in part on information from the additional data about the one or more target attributes of that object; and

providing, by the one or more computing devices, information about the determined assessment of the multiple objects.

16. The non-transitory computer-readable medium of claim 15 wherein the stored contents include software instructions that, when executed, cause the one or more computing devices to perform further automated operations that include determining, by the one or more computing devices and based at least in part on the analyzing of the visual data of the one or more images, that the visual data lacks a level of details to satisfy a defined threshold about the one or more target attributes of each of the multiple objects, and wherein the providing of the instructions is performed in response to the determining that the visual data lacks the level of details.

17. The non-transitory computer-readable medium of claim 15 wherein the automated operations further include determining, by the one or more computing devices, an assessment of the usability of the room for the indicated purpose, including combining information about the assessments of the contributions of the multiple objects, and wherein the providing of the information about the determined assessment for each of the multiple objects further includes displaying, by the one or more computing devices, information about the determined assessment of the usability of the room.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 11, 2023
From: STOEVA, VIKTORIYA; KANG, SING BING; KHOSRAVAN, NAJI; WIXSON, LAMBERT E.
To: ZILLOW GROUP, INC.
Reel/Frame 064854/0232 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 11, 2023
From: ZILLOW GROUP, INC.
To: ZILLOW, INC.
Reel/Frame 064859/0518 →
MERGER AND CHANGE OF NAME Recorded Sep 11, 2023
From: PUSH SUB I, INC.; MFTB HOLDCO, INC.
To: MFTB HOLDCO, INC.
Reel/Frame 064859/0815 →
ENTITY CONVERSION Recorded Sep 11, 2023
From: ZILLOW, INC.
To: ZILLOW LLC
Reel/Frame 064867/0168 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 11, 2023
From: ZILLOW LLC
To: PUSH SUB I, INC.
Reel/Frame 064867/0173 →
Continuity (2)
Continuation 17185793 · Feb 25, 2021
Related Publication 20230419662A1 · Dec 28, 2023
References Cited (301)
US 5140352A · Moore et al. · 1992 [cited by applicant]
US 6031540A · Golin et al. · 2000 [cited by applicant]
US 6141034A · McCutchen · 2000 [cited by applicant]
US 6317166B1 · McCutchen · 2001 [cited by applicant]
US 6320584B1 · Golin et al. · 2001 [cited by applicant]
US 6323858B1 · Gilbert et al. · 2001 [cited by applicant]
US 6337683B1 · Gilbert et al. · 2002 [cited by applicant]
US 6654019B2 · Gilbert et al. · 2003 [cited by applicant]
US 6683608B2 · Golin et al. · 2004 [cited by applicant]
US 6690374B2 · Park et al. · 2004 [cited by applicant]
US 6731305B1 · Park et al. · 2004 [cited by applicant]
US 6738073B2 · Park et al. · 2004 [cited by applicant]
US 7050085B1 · Park et al. · 2006 [cited by applicant]
US 7129971B2 · McCutchen · 2006 [cited by applicant]
US 7196722B2 · White et al. · 2007 [cited by applicant]
US 7522186B2 · Arpa · 2009 [cited by examiner]
US 7525567B2 · McCutchen · 2009 [cited by applicant]
US 7620909B2 · Park et al. · 2009 [cited by applicant]
US 7627235B2 · McCutchen et al. · 2009 [cited by applicant]
US 7660431B2 · Crenshaw · 2010 [cited by examiner]
US 7782319B2 · Ghosh et al. · 2010 [cited by applicant]
US 7791638B2 · McCutchen · 2010 [cited by applicant]
US 7909241B2 · Stone et al. · 2011 [cited by applicant]
US 7973838B2 · McCutchen · 2011 [cited by applicant]
US 8072455B2 · Temesvari et al. · 2011 [cited by applicant]
US 8094182B2 · Park et al. · 2012 [cited by applicant]
US RE43786E · Cooper · 2012 [cited by applicant]
US 8463020B1 · Schuckmann et al. · 2013 [cited by applicant]
US 8517256B2 · Stone et al. · 2013 [cited by applicant]
US 8520060B2 · Zomet et al. · 2013 [cited by applicant]
US 8523066B2 · Stone et al. · 2013 [cited by applicant]
US 8523067B2 · Stone et al. · 2013 [cited by applicant]
US 8528816B2 · Stone et al. · 2013 [cited by applicant]
US 8540153B2 · Stone et al. · 2013 [cited by applicant]
US 8594428B2 · Aharoni et al. · 2013 [cited by applicant]
US 8654180B2 · Zomet et al. · 2014 [cited by applicant]
US 8666815B1 · Chau · 2014 [cited by applicant]
US 8699005B2 · Likholyot · 2014 [cited by applicant]
US 8705892B2 · Aguilera et al. · 2014 [cited by applicant]
US RE44924E · Cooper et al. · 2014 [cited by applicant]
US 8854684B2 · Zomet · 2014 [cited by applicant]
US 8861840B2 · Bell et al. · 2014 [cited by applicant]
US 8861841B2 · Bell et al. · 2014 [cited by applicant]
US 8879828B2 · Bell et al. · 2014 [cited by applicant]
US 8953871B2 · Zomet · 2015 [cited by applicant]
US 8989440B2 · Klusza et al. · 2015 [cited by applicant]
US 8996336B2 · Malka et al. · 2015 [cited by applicant]
US 9021947B2 · Landa · 2015 [cited by applicant]
US 9026947B2 · Lee et al. · 2015 [cited by applicant]
US 9035968B2 · Zomet · 2015 [cited by applicant]
US 9041796B2 · Malka et al. · 2015 [cited by applicant]
US 9071714B2 · Zomet · 2015 [cited by applicant]
US 9129438B2 · Aarts et al. · 2015 [cited by applicant]
US 9151608B2 · Malka et al. · 2015 [cited by applicant]
US 9165410B1 · Bell et al. · 2015 [cited by applicant]
US 9171405B1 · Bell et al. · 2015 [cited by applicant]
US 9324190B2 · Bell et al. · 2016 [cited by applicant]
US 9361717B2 · Zomet · 2016 [cited by applicant]
US 9396586B2 · Bell et al. · 2016 [cited by applicant]
US 9438759B2 · Zomet · 2016 [cited by applicant]
US 9438775B2 · Powers et al. · 2016 [cited by applicant]
US 9489775B1 · Bell et al. · 2016 [cited by applicant]
US 9495783B1 · Samarasekera et al. · 2016 [cited by applicant]
US 9576401B2 · Zomet · 2017 [cited by applicant]
US 9619933B2 · Spinella-Marno et al. · 2017 [cited by applicant]
US 9635252B2 · Accardo et al. · 2017 [cited by applicant]
US 9641702B2 · Bin-Nun et al. · 2017 [cited by applicant]
US 9760994B1 · Bell et al. · 2017 [cited by applicant]
US 9786097B2 · Bell et al. · 2017 [cited by applicant]
US 9787904B2 · Birkler et al. · 2017 [cited by applicant]
US 9836885B1 · Eraker et al. · 2017 [cited by applicant]
US 9852351B2 · Aguilera Perez et al. · 2017 [cited by applicant]
US 9953111B2 · Bell et al. · 2018 [cited by applicant]
US 9953430B1 · Zakhor · 2018 [cited by applicant]
US 9990760B2 · Aguilera Perez et al. · 2018 [cited by applicant]
US 9990767B1 · Sheffield et al. · 2018 [cited by applicant]
US 10026224B2 · Bell et al. · 2018 [cited by applicant]
US 10030979B2 · Bjorke et al. · 2018 [cited by applicant]
US 10055876B2 · Ford et al. · 2018 [cited by applicant]
US 10068344B2 · Jovanovic et al. · 2018 [cited by applicant]
US 10083522B2 · Jovanovic et al. · 2018 [cited by applicant]
US 10102639B2 · Bell et al. · 2018 [cited by applicant]
US 10102673B2 · Eraker et al. · 2018 [cited by applicant]
US 10120397B1 · Zakhor et al. · 2018 [cited by applicant]
US 10122997B1 · Sheffield et al. · 2018 [cited by applicant]
US 10127718B2 · Zakhor et al. · 2018 [cited by applicant]
US 10127722B2 · Shakib et al. · 2018 [cited by applicant]
US 10139985B2 · Mildrew et al. · 2018 [cited by applicant]
US 10163261B2 · Bell et al. · 2018 [cited by applicant]
US 10163271B1 · Powers et al. · 2018 [cited by applicant]
US 10181215B2 · Sedeffow · 2019 [cited by applicant]
US 10192115B1 · Sheffield et al. · 2019 [cited by applicant]
US 10204185B2 · Mrowca et al. · 2019 [cited by applicant]
US 10210285B2 · Wong et al. · 2019 [cited by applicant]
US 10235797B1 · Sheffield et al. · 2019 [cited by applicant]
US 10242400B1 · Eraker et al. · 2019 [cited by applicant]
US 10339716B1 · Powers et al. · 2019 [cited by applicant]
US 10366531B2 · Sheffield · 2019 [cited by applicant]
US 10375306B2 · Shan et al. · 2019 [cited by applicant]
US 10395435B2 · Powers et al. · 2019 [cited by applicant]
US 10530997B2 · Shan et al. · 2020 [cited by applicant]
US 10643386B2 · Li et al. · 2020 [cited by applicant]
US 10708507B1 · Dawson et al. · 2020 [cited by applicant]
US 10809066B2 · Colburn et al. · 2020 [cited by applicant]
US 10825247B1 · Vincent et al. · 2020 [cited by applicant]
US 10834317B2 · Boyadzhiev et al. · 2020 [cited by applicant]
US 11055531B1 · Maestas · 2021 [cited by applicant]
US 11057561B2 · Shan et al. · 2021 [cited by applicant]
US 11164361B2 · Moulon et al. · 2021 [cited by applicant]
US 11164368B2 · Vincent et al. · 2021 [cited by applicant]
US 11165959B2 · Shan et al. · 2021 [cited by applicant]
US 11200421B1 · Chavez et al. · 2021 [cited by applicant]
US 11217019B2 · Li et al. · 2022 [cited by applicant]
US 11238652B2 · Impas et al. · 2022 [cited by applicant]
US 11243656B2 · Li et al. · 2022 [cited by applicant]
US 11252329B1 · Cier et al. · 2022 [cited by applicant]
US 11284006B2 · Dawson et al. · 2022 [cited by applicant]
US 11405549B2 · Cier et al. · 2022 [cited by applicant]
US 11405558B2 · Dawson et al. · 2022 [cited by applicant]
US 11408738B2 · Colburn et al. · 2022 [cited by applicant]
US 11480433B2 · Colburn et al. · 2022 [cited by applicant]
US 11481925B1 · Li et al. · 2022 [cited by applicant]
US 11494973B2 · Boyadzhiev et al. · 2022 [cited by applicant]
US 11501492B1 · Li et al. · 2022 [cited by applicant]
US 11514674B2 · Moulon et al. · 2022 [cited by applicant]
US 11592969B2 · Li et al. · 2023 [cited by applicant]
US 11632516B2 · Boyadzhiev et al. · 2023 [cited by applicant]
US 11632602B2 · Cier et al. · 2023 [cited by applicant]
US 11638069B2 · Guan et al. · 2023 [cited by applicant]
US 11645781B2 · Li et al. · 2023 [cited by applicant]
US 11676344B2 · Li et al. · 2023 [cited by applicant]
US 11790648B2 · Stoeva · 2023 [cited by examiner]
US 11836973B2 · Wixson · 2023 [cited by examiner]
US 20060256109A1 · Acker et al. · 2006 [cited by applicant]
US 20100232709A1 · Zhang et al. · 2010 [cited by applicant]
US 20120075414A1 · Park et al. · 2012 [cited by applicant]
US 20120293613A1 · Powers et al. · 2012 [cited by applicant]
US 20130050407A1 · Brinda et al. · 2013 [cited by applicant]
US 20130342533A1 · Bell et al. · 2013 [cited by applicant]
US 20140043436A1 · Bell et al. · 2014 [cited by applicant]
US 20140044343A1 · Bell et al. · 2014 [cited by applicant]
US 20140044344A1 · Bell et al. · 2014 [cited by applicant]
US 20140125658A1 · Bell et al. · 2014 [cited by applicant]
US 20140125767A1 · Bell et al. · 2014 [cited by applicant]
US 20140125768A1 · Bell et al. · 2014 [cited by applicant]
US 20140125769A1 · Bell et al. · 2014 [cited by applicant]
US 20140125770A1 · Bell et al. · 2014 [cited by applicant]
US 20140236482A1 · Dorum et al. · 2014 [cited by applicant]
US 20140267631A1 · Powers et al. · 2014 [cited by applicant]
US 20140307100A1 · Myllykoski et al. · 2014 [cited by applicant]
US 20140320674A1 · Kuang · 2014 [cited by applicant]
US 20150109338A1 · McKinnon et al. · 2015 [cited by applicant]
US 20150116691A1 · Likholyot · 2015 [cited by applicant]
US 20150189165A1 · Milosevski et al. · 2015 [cited by applicant]
US 20150262421A1 · Bell et al. · 2015 [cited by applicant]
US 20150269785A1 · Bell et al. · 2015 [cited by applicant]
US 20150302636A1 · Arnoldus et al. · 2015 [cited by applicant]
US 20150310596A1 · Sheridan et al. · 2015 [cited by applicant]
US 20150332464A1 · O'Keefe et al. · 2015 [cited by applicant]
US 20160055268A1 · Bell et al. · 2016 [cited by applicant]
US 20160134860A1 · Jovanovic et al. · 2016 [cited by applicant]
US 20160140676A1 · Fritze et al. · 2016 [cited by applicant]
US 20160217225A1 · Bell et al. · 2016 [cited by applicant]
US 20160260250A1 · Jovanovic et al. · 2016 [cited by applicant]
US 20160275518A1 · Bowles · 2016 [cited by examiner]
US 20160286119A1 · Rondinelli · 2016 [cited by applicant]
US 20160300385A1 · Bell et al. · 2016 [cited by applicant]
US 20170034430A1 · Fu et al. · 2017 [cited by applicant]
US 20170067739A1 · Siercks et al. · 2017 [cited by applicant]
US 20170194768A1 · Powers et al. · 2017 [cited by applicant]
US 20170195654A1 · Powers et al. · 2017 [cited by applicant]
US 20170263050A1 · Ha et al. · 2017 [cited by applicant]
US 20170324941A1 · Birkler · 2017 [cited by applicant]
US 20170330273A1 · Holt et al. · 2017 [cited by applicant]
US 20170337737A1 · Edwards et al. · 2017 [cited by applicant]
US 20180007340A1 · Stachowski · 2018 [cited by applicant]
US 20180025536A1 · Bell et al. · 2018 [cited by applicant]
US 20180075168A1 · Tiwari et al. · 2018 [cited by applicant]
US 20180139431A1 · Simek et al. · 2018 [cited by applicant]
US 20180143023A1 · Bjorke et al. · 2018 [cited by applicant]
US 20180143756A1 · Mildrew et al. · 2018 [cited by applicant]
US 20180144487A1 · Bell et al. · 2018 [cited by applicant]
US 20180144535A1 · Ford et al. · 2018 [cited by applicant]
US 20180144547A1 · Shakib et al. · 2018 [cited by applicant]
US 20180144555A1 · Ford et al. · 2018 [cited by applicant]
US 20180146121A1 · Hensler et al. · 2018 [cited by applicant]
US 20180146193A1 · Safreed et al. · 2018 [cited by applicant]
US 20180146212A1 · Hensler et al. · 2018 [cited by applicant]
US 20180165871A1 · Mrowca · 2018 [cited by applicant]
US 20180203955A1 · Bell et al. · 2018 [cited by applicant]
US 20180241985A1 · O'Keefe et al. · 2018 [cited by applicant]
US 20180293793A1 · Bell et al. · 2018 [cited by applicant]
US 20180300936A1 · Ford et al. · 2018 [cited by applicant]
US 20180306588A1 · Bjorke et al. · 2018 [cited by applicant]
US 20180348854A1 · Powers et al. · 2018 [cited by applicant]
US 20180365496A1 · Hovden et al. · 2018 [cited by applicant]
US 20190012833A1 · Eraker et al. · 2019 [cited by applicant]
US 20190026956A1 · Gausebeck et al. · 2019 [cited by applicant]
US 20190026957A1 · Gausebeck · 2019 [cited by applicant]
US 20190026958A1 · Gausebeck et al. · 2019 [cited by applicant]
US 20190035165A1 · Gausebeck · 2019 [cited by applicant]
US 20190041972A1 · Bae · 2019 [cited by applicant]
US 20190050137A1 · Mildrew et al. · 2019 [cited by applicant]
US 20190051050A1 · Bell et al. · 2019 [cited by applicant]
US 20190051054A1 · Jovanovic et al. · 2019 [cited by applicant]
US 20190087067A1 · Hovden et al. · 2019 [cited by applicant]
US 20190122422A1 · Sheffield et al. · 2019 [cited by applicant]
US 20190164335A1 · Sheffield et al. · 2019 [cited by applicant]
US 20190180104A1 · Sheffield et al. · 2019 [cited by applicant]
US 20190251645A1 · Winans · 2019 [cited by applicant]
US 20190266293A1 · Shida et al. · 2019 [cited by applicant]
US 20190287164A1 · Eraker et al. · 2019 [cited by applicant]
US 20200074668A1 · Stenger et al. · 2020 [cited by applicant]
US 20200302681A1 · Totty et al. · 2020 [cited by applicant]
US 20200336675A1 · Dawson et al. · 2020 [cited by applicant]
US 20200389602A1 · Dawson et al. · 2020 [cited by applicant]
US 20200408532A1 · Colburn et al. · 2020 [cited by applicant]
US 20210004589A1 · Turkelson et al. · 2021 [cited by applicant]
US 20210021761A1 · Shan et al. · 2021 [cited by applicant]
US 20210027485A1 · Zhang · 2021 [cited by applicant]
US 20210044760A1 · Dawson et al. · 2021 [cited by applicant]
US 20210065271A1 · Benkreira et al. · 2021 [cited by applicant]
US 20210117582A1 · Kamenca et al. · 2021 [cited by applicant]
US 20210199809A1 · Wynn · 2021 [cited by applicant]
US 20210377442A1 · Boyadzhiev et al. · 2021 [cited by applicant]
US 20210385378A1 · Cier et al. · 2021 [cited by applicant]
US 20220003555A1 · Colburn et al. · 2022 [cited by applicant]
US 20220028156A1 · Boyadzhiev et al. · 2022 [cited by applicant]
US 20220028159A1 · Vincent et al. · 2022 [cited by applicant]
US 20220076019A1 · Moulon et al. · 2022 [cited by applicant]
US 20220076491A1 · Zellner et al. · 2022 [cited by applicant]
US 20220092227A1 · Yin et al. · 2022 [cited by applicant]
US 20220114291A1 · Li et al. · 2022 [cited by applicant]
US 20220164493A1 · Li et al. · 2022 [cited by applicant]
US 20220189122A1 · Li et al. · 2022 [cited by applicant]
EP 2413097A2 · 2012 [cited by applicant]
EP 2505961A2 · 2012 [cited by applicant]
EP 2506170A2 · 2012 [cited by applicant]
KR 101770648B1 · 2017 [cited by applicant]
KR 101930796B1 · 2018 [cited by applicant]
WO 2005091894A2 · 2005 [cited by applicant]
WO 2016154306A1 · 2016 [cited by applicant]
WO 2018170421A1 · 2018 [cited by applicant]
WO 2018204279A1 · 2018 [cited by applicant]
WO 2019058266A1 · 2019 [cited by applicant]
WO 2019083832A1 · 2019 [cited by applicant]
WO 2019104049A1 · 2019 [cited by applicant]
WO 2019118599A2 · 2019 [cited by applicant]
WO 2020068177A1 · 2020 [cited by applicant]
WO 2022046358A1 · 2022 [cited by applicant]
WO 2022053888A1 · 2022 [cited by applicant]
WO 2022101707A1 · 2022 [cited by applicant]
CubiCasa | From video to floor plan in under 5 minutes, retrieved on Mar. 26, 2019, from https://www.cubi.casa/, 6 pages. [cited by applicant]
CubiCasa FAQ & Manual, retrieved on Mar. 26, 2019, from https://www.cubi.casa/faq/, 5 pages. [cited by applicant]
Cupix Home, retrieved on Mar. 26, 2019, from https://www.cupix.com/, 1 page. [cited by applicant]
Cupix—FAQ, retrieved on Mar. 26, 2019, from https://www.cupix.com/faq.html, 3 pages. [cited by applicant]
IGUIDE: 3D Virtual Tours, retrieved on Mar. 26, 2019, from https://goiguide.com/, 6 pages. [cited by applicant]
immoviewer.com | Automated Video Creation & Simple Affordable 3D 360 Tours, retrieved on Mar. 26, 2019, from https://www.immoviewer.com/, 5 pages. [cited by applicant]
MagicPlan | #1 Floor Plan App, Construction & Surveying Samples, retrieved on Mar. 26, 2019, from https://www.magicplan.app/, 9 pages. [cited by applicant]
EyeSpy360 Virtual Tours | Virtual Tour with any 360 camera, retrieved on Mar. 27, 2019, from https://www.eyespy360.com/en-us/, 15 pages. [cited by applicant]
Indoor Reality, retrieved on Mar. 27, 2019, from https://www.indoorreality.com/, 9 pages. [cited by applicant]
Insidemaps, retrieved on Mar. 27, 2019, from https://www.insidemaps.com/, 7 pages. [cited by applicant]
IStaging | Augmented & Virtual Reality Platform for Business, retrieved on Mar. 27, 2019, from https://www.istaging.com/en/, 7 pages. [cited by applicant]
Metareal, retrieved on Mar. 27, 2019, from https://www.metareal.com/, 4 pages. [cited by applicant]
PLNAR—The Ar 3D Measuring / Modeling Platform, retrieved on Mar. 27, 2019, from https://www.plnar.co, 6 pages. [cited by applicant]
YouVR Global, retrieved on Mar. 27, 2019, from https://global.youvr.io/, 9 pages. [cited by applicant]
GeoCV, retrieved on Mar. 28, 2019, from https://geocv.com/, 4 pages. [cited by applicant]
Biersdorfer, J.D., “How to Make A 3-D Model of Your Home Renovation Vision,” in The New York Times, Feb. 13, 2019, retrieved Mar. 28, 2019, 6 pages. [cited by applicant]
Chen et al. “Rise of the indoor crowd: Reconstruction of building interior view via mobile crowdsourcing.” In: Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. Nov. 4, 2015, 13 pages. [cited by applicant]
Immersive 3D for the Real World, retrieved from https://matterport.com/, on Mar. 27, 2017, 5 pages. [cited by applicant]
Learn About Our Complete 3D System, retrieved from https://matterport.com/how-it-works/, on Mar. 27, 2017, 6 pages. [cited by applicant]
Surefield FAQ, retrieved from https://surefield.com/faq, on Mar. 27, 2017, 1 page. [cited by applicant]
Why Surefield, retrieved from https://surefield.com/why-surefield, on Mar. 27, 2017, 7 pages. [cited by applicant]
Schneider, V., “Create immersive photo experiences with Google Photo Sphere,” retrieved from http://geojoumalism.org/2015/02/create-immersive-photo-experiences-with-google-photo-sphere/, on Mar. 27, 2017, 7 pages. [cited by applicant]
Tango (platform), Wikipedia, retrieved from https://en.wikipedia.org/wiki/Tango_(platform), on Jun. 12, 2018, 5 pages. [cited by applicant]
Zou et al. “LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image” in arXiv:1803.08999, submitted Mar. 23, 2018, 9 pages. [cited by applicant]
Lee et al. “RoomNet: End-to-End Room Layout Estimation” in arXiv:1703.00241v2, submitted Aug. 7, 2017, 10 pages. [cited by applicant]
Time-of-flight camera, Wikipedia, retrieved from https://en.wikipedia.org/wiki/Time-of-flight_camera, on Aug. 30, 2018, 8 pages. [cited by applicant]
Magicplan—Android Apps on Go . . . , retrieved from https://play.google.com/store/apps/details?id=com.sensopia.magicplan, on Feb. 21, 2018, 5 pages. [cited by applicant]
Pintore et al., “AtlantaNet: Inferring the 3D Indoor Layout from a Single 360 Image beyond the Manhattan World Assumption”, ECCV 2020, 16 pages. [cited by applicant]
Cowles, Jeremy, “Differentiable Rendering”, Aug. 19, 2018, accessed Dec. 7, 2020 at https://towardsdatascience.com/differentiable-rendering-d00a4b0f14be, 3 pages. [cited by applicant]
Yang et al., “DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama”, in arXiv:1811.11977[cs.v2], submitted Apr. 2, 2019, 14 pages. [cited by applicant]
Sun et al., “HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features”, in arXiv:2011.11498[cs.v2], submitted Nov. 24, 2020, 15 pages. [cited by applicant]
Nguyen-Phuoc et al., “RenderNet: A deep convolutional network for differentiable rendering from 3D shapes”, in arXiv:1806.06575[cs.v3], submitted Apr. 1, 2019, 17 pages. [cited by applicant]
Convolutional neural network, Wikipedia, retrieved from https://en.wikipedia.org/wiki/Convolutional_neural_network, on Dec. 7, 2020, 25 pages. [cited by applicant]
Hamilton et al., “Inductive Representation Learning on Large Graphs”, in 31st Conference on Neural Information Processing Systems (NIPS 2017), 2017, 19 pages. [cited by applicant]
Kipf et al., “Variational Graph Auto-Encoders”, in arXiv:1611.07308v1 [stat.ML], submitted Nov. 21, 2016, 3 pages. [cited by applicant]
Cao et al., “MolGAN: An Implicit Generative Model for Small Molecular Graphs”, in arXiv:1805.11973v1 [stat.ML], submitted May 30, 2018, 11 pages. [cited by applicant]
Chen et al., “Intelligent Home 3D: Automatic 3D-House Design from Linguistic Descriptions Only”, in arXiv:2003.00397v1 [cs.CV], submitted Mar. 1, 2020, 14 pages. [cited by applicant]
Cucurull et al., “Context-Aware Visual Compatibility Prediction”, in arXiv:1902.03646v2 [cs.CV], submitted Feb. 12, 2019, 10 pages. [cited by applicant]
Fan et al., “Labeled Graph Generative Adversarial Networks”, in arXiv:1906.03220v1 [cs.LG], submitted Jun. 7, 2019, 14 pages. [cited by applicant]
Gong et al., “Exploiting Edge Features in Graph Neural Networks”, in arXiv: 1809.02709v2 [cs.LG], submitted Jan. 28, 2019, 10 pages. [cited by applicant]
Genghis Goodman, “A Machine Learning Approach to Artificial Floorplan Generation”, University of Kentucky Theses and Dissertations-Computer Science, 2019, accessible at https://uknowledge.uky.edu/cs_etds/89, 40 pages. [cited by applicant]
Grover et al., “node2vec: Scalable Feature Learning for Networks”, in arXiv:1607.00653v1 [cs.SI], submitted Jul. 3, 2016, 10 pages. [cited by applicant]
Nauata et al., “House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation”, in arXiv:2003.06988v1 [cs.CV], submitted Mar. 16, 2020, 17 pages. [cited by applicant]
Kang et al., “A Review of Techniques for 3D Reconstruction of Indoor Environments”, in ISPRS International Journal oOf Geo-Information 2020, May 19, 2020, 31 pages. [cited by applicant]
Kipf et al., “Semi-Supervised Classification With Graph Convolutional Networks”, in arXiv:1609.02907v4 [cs.LG], submitted Feb. 22, 2017, 14 pages. [cited by applicant]
Li et al., “Graph Matching Networks for Learning the Similarity of Graph Structured Objects”, in Proceedings of the 36th International Conference on Machine Learning (PMLR 97), 2019, 18 pages. [cited by applicant]
Liu et al., “Hyperbolic Graph Neural Networks”, in 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), 2019, 12 pages. [cited by applicant]
Merrell et al., “Computer-Generated Residential Building Layouts”, in ACM Transactions on Graphics, Dec. 2010, 13 pages. [cited by applicant]
Kim et al., “Room Layout Estimation With Object and Material Attributes Information Using a Spherical Camera”, 2016 ,Fourth International Conference on 3D Vision (3DV), Oct. 25-28, 2016, 9 pages. [cited by applicant]
Cited By (1)
US 12,505,618