IP Library › Granted Patent US 12,299,843
Granted Patent B2
US 12,299,843 · App. 18/382,393 · Granted May 13, 2025

Systems and methods for three-dimensional environmental modeling of a particular location such as a commercial or residential property

Inventor: Mark Thomas (Woodstock, GA)
Assignee: EchoSense, LLC
G06T5/50G06T7/70G06T15/503G06T2207/20221G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,299,843
App. No.
18/382,393
Granted
May 13, 2025
Kind
B2
Abstract

A computer-implemented method, according to various embodiments, includes receiving images such as photographs and videos from one or more portable computing devices associated with one or more individuals (e.g., construction workers or landscapers) while the portable computing devices are in a particular position within a particular location at a particular time. The method further includes determining a virtual position within a representation of the particular location that generally corresponds to the particular position and combining the images with the representation to generate an enhanced representation of the particular location. This may allow, for example, owners of a particular property to track and quickly understand construction and landscaping work that has been done on their property and to easily contact those individuals regarding that work.

Claims (65)

1. A method of generating an enhanced representation of a particular property, the method comprising the steps of:

i. receiving, by at least one computer processor, a first image taken in a first position within a particular location on the particular property;

ii. receiving, by at least one computer processor, a second image taken in a second position within the particular location on the particular property;

iii. determining, by at least one computer processor, a first virtual position within a 3-D representation of the particular property that generally corresponds to the first position;

iv. determining, by at least one computer processor, a second virtual position within the 3-D representation of the particular property that generally corresponds to the second position;

v. after receiving the first and second images, combining, by at least one computer processor, the first image, the second image, and the 3-D representation so that the first position of the first image generally corresponds to the first virtual position within the 3-D representation and the second position of the second image generally corresponds to the second virtual position within the 3-D representation; and

vi. at least partially in response to combining the first image, the second image, and the 3-D representation, generating, by at least one computer processor, an enhanced representation of the particular property that includes the first image, the second image, and the 3-D representation of the particular property, wherein:

1. combining the first image, the second image, and the 3-D representation comprises combining a portion of the first image with a portion of the second image; and

2. the enhanced representation of the particular property includes the portion of the first image, the portion of the second image, and the 3-D representation of the particular property, wherein:

a. the portion of the first image comprises less than all of the first image, and the portion of the second image comprises less than all of the second image.

2. The method of claim 1 , wherein the first image and the second image were taken at different times.

3. The method of claim 2 , wherein:

the first image is a 2-D image of a first particular construction element associated with the particular property;

the second image is a 2-D image of a second particular construction element associated with the particular property; and

the enhanced representation of the particular property is an enhanced 3-D representation of the particular property that includes the first and second images.

4. The method of claim 2 , further comprising:

converting the first image into a first 3-D video image;

converting the second image into a second 3-D video image; and

generating a virtual reality representation of the particular location by superimposing the first 3-D video image and the second 3-D video image over the 3-D representation of the particular property.

5. The method of claim 4 , wherein the 3-D representation of the particular property comprises a 3-D animation of the particular property.

6. The method of claim 2 , wherein combining the first image and the 3-D representation comprises identifying one or more matching objects from within each of the first and second images, removing the corresponding one or more objects from the first image or the second image, and inserting the remaining portions of the first or second images into the 3-D representation of the particular property.

7. The method of claim 1 , wherein the first image is a photo image or video image.

8. A method of generating an enhanced representation of a particular property, the method comprising the steps of:

i. receiving, by at least one computer processor, a first image taken in a first position within a particular location on the particular property;

ii. receiving, by at least one computer processor, a second image taken in a second position within the particular location on the particular property;

iii determining, by at least one computer processor, a first virtual position within a representation of the particular property that generally corresponds to the first position;

iv. determining, by at least one computer processor, a second virtual position within the representation of the particular property that generally corresponds to the second position;

v. after receiving the first and second images, combining, by at least one computer processor, the first image, the second image, and the representation so that the first position of the first image generally corresponds to the first virtual position within the representation and the second position of the second image generally corresponds to the second virtual position within the representation; and

vi. at least partially in response to combining the first image, the second image, and the representation, generating, by at least one computer processor, an enhanced representation of the particular property that includes the first image, the second image, and the representation of the particular property, wherein:

1. combining the first image, the second image, and the representation comprises combining a portion of the first image with a portion of the second image; and

2. the enhanced representation of the particular property includes the portion of the first image, the portion of the second image, and the representation of the particular property, wherein:

a. the portion of the first image comprises less than all of the first image, and the portion of the second image comprises less than all of the second image.

9. The method of claim 8 , wherein the first image and the second image were taken at different times.

10. The method of claim 9 , wherein:

the first image is a 2-D image of a first particular construction element associated with the particular property;

the second image is a 2-D image of a second particular construction element associated with the particular property; and

the enhanced representation of the particular property is an enhanced representation of the particular property that includes the first and second images.

11. The method of claim 9 , further comprising:

converting the first image into a first video image;

converting the second image into a second video image; and

generating a virtual reality representation of the particular location by superimposing the first video image and the second video image over the representation of the particular property.

12. The method of claim 11 , wherein the representation of the particular property comprises an animation of the particular property.

13. The method of claim 9 , wherein combining the first image and the representation comprises identifying one or more matching objects from within each of the first and second images, removing the corresponding one or more objects from the first image or the second image, and inserting the remaining portions of the first or second images into the representation of the particular property.

14. The method of claim 8 , wherein the first image is a photo image or video image.

15. A non-transitory computer-readable medium storing computer-executable instructions for:

i. receiving, by at least one computer processor, a first image taken in a first position within a particular location on the particular property;

ii. receiving, by at least one computer processor, a second image taken in a second position within the particular location on the particular property;

iii. determining, by at least one computer processor, a first virtual position within a representation of the particular property that generally corresponds to the first position;

iv. determining, by at least one computer processor, a second virtual position within the representation of the particular property that generally corresponds to the second position;

v. after receiving the first and second images, combining, by at least one computer processor, the first image, the second image, and the representation so that the first position of the first image generally corresponds to the first virtual position within the representation and the second position of the second image generally corresponds to the second virtual position within the representation; and

vi. at least partially in response to combining the first image, the second image, and the representation, generating, by at least one computer processor, an enhanced representation of the particular property that includes the first image, the second image, and the representation of the particular property, wherein:

1. combining the first image, the second image, and the representation comprises combining a portion of the first image with a portion of the second image; and

2. the enhanced representation of the particular property includes the portion of the first image, the portion of the second image, and the representation of the particular property, wherein:

a. the portion of the first image comprises less than all of the first image, and the portion of the second image comprises less than all of the second image.

16. The non-transitory computer-readable medium of claim 15 , wherein the first image and the second image were taken at different times.

17. The non-transitory computer-readable medium of claim 16 , wherein:

the first image is a 2-D image of a first particular construction element associated with the particular property;

the second image is a 2-D image of a second particular construction element associated with the particular property; and

the enhanced representation of the particular property is an enhanced representation of the particular property that includes the first and second images.

18. The non-transitory computer-readable medium of claim 16 , wherein the computer-readable medium stores computer-executable instructions for:

converting the first image into a first video image;

converting the second image into a second video image; and

generating a virtual reality representation of the particular location by superimposing the first video image and the second video image over the representation of the particular property.

19. The non-transitory computer-readable medium of claim 18 , wherein the representation of the particular property comprises an animation of the particular property.

20. The non-transitory computer-readable medium of claim 16 , wherein combining the first image and the representation comprises identifying one or more matching objects from within each of the first and second images, removing the corresponding one or more objects from the first image or the second image, and inserting the remaining portions of the first or second images into the representation of the particular property.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2023
From: THOMAS, MARK
To: QUASAR BLU, LLC
Reel/Frame 065309/0837 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2023
From: QUASAR BLU, LLC
To: ECHOSENSE, LLC
Reel/Frame 065309/0967 →
Continuity (7)
Continuation 17397211 · Aug 9, 2021
Continuation 16833798 · Mar 30, 2020
Continuation In Part 16457569 · Jun 28, 2019
Continuation In Part 15973286 · May 7, 2018
Continuation 15367923 · Dec 2, 2016
Provisional Application 62262879 · Dec 3, 2015
Related Publication 20240046420A1 · Feb 8, 2024
References Cited (69)
US 7034861B2 · Okada et al. · 2006 [cited by applicant]
US 7548833B2 · Ahmed · 2009 [cited by applicant]
US 7693702B1 · Kerner et al. · 2010 [cited by applicant]
US 7909696B2 · Beaulieu et al. · 2011 [cited by applicant]
US 8032481B2 · Pinckney et al. · 2011 [cited by applicant]
US 8131549B2 · Teegan et al. · 2012 [cited by applicant]
US 8162758B1 · Powers et al. · 2012 [cited by applicant]
US 8317613B2 · Lerner et al. · 2012 [cited by applicant]
US 8548308B2 · Okubo et al. · 2013 [cited by applicant]
US 8620146B1 · Coleman · 2013 [cited by applicant]
US 8624974B2 · Tsai et al. · 2014 [cited by applicant]
US 8666909B2 · Pinckney et al. · 2014 [cited by applicant]
US 9032297B2 · Lovejoy et al. · 2015 [cited by applicant]
US 9135955B2 · Abecassis · 2015 [cited by applicant]
US 9159034B2 · Pinckney et al. · 2015 [cited by applicant]
US 9201974B2 · Arrasvuori et al. · 2015 [cited by applicant]
US 9235920B2 · Girdzijauskas et al. · 2016 [cited by applicant]
US 9405432B2 · Vats et al. · 2016 [cited by applicant]
US 9449106B2 · Reese et al. · 2016 [cited by applicant]
US 9563984B2 · Willis et al. · 2017 [cited by applicant]
US 9646025B2 · Boyns et al. · 2017 [cited by applicant]
US 9965837B1 · Thomas · 2018 [cited by applicant]
US 10339644B2 · Thomas et al. · 2019 [cited by applicant]
US 10607328B2 · Thomas · 2020 [cited by applicant]
US 11087445B2 · Thomas · 2021 [cited by applicant]
US 11798148B2 · Thomas · 2023 [cited by examiner]
US 20040104935A1 · Williamson et al. · 2004 [cited by applicant]
US 20060186197A1 · Rosenberg · 2006 [cited by applicant]
US 20070118805A1 · Kraus et al. · 2007 [cited by applicant]
US 20070168208A1 · Aikas et al. · 2007 [cited by applicant]
US 20090238378A1 · Kikinis et al. · 2009 [cited by applicant]
US 20100161432A1 · Kumanov et al. · 2010 [cited by applicant]
US 20110022457A1 · Carney, II et al. · 2011 [cited by applicant]
US 20110149022A1 · Klebanov et al. · 2011 [cited by applicant]
US 20120323626A1 · Beauregard et al. · 2012 [cited by applicant]
US 20120330734A1 · Brown et al. · 2012 [cited by applicant]
US 20130162766A1 · Cohen · 2013 [cited by applicant]
US 20130218531A1 · Deichmann et al. · 2013 [cited by applicant]
US 20130237304A1 · Oakes et al. · 2013 [cited by applicant]
US 20140089223A1 · Klassen et al. · 2014 [cited by applicant]
US 20140132633A1 · Fekete et al. · 2014 [cited by applicant]
US 20140133832A1 · Sumler et al. · 2014 [cited by applicant]
US 20140156319A1 · Deeb · 2014 [cited by applicant]
US 20140278403A1 · Jacob et al. · 2014 [cited by applicant]
US 20140358703A1 · Stuntebeck et al. · 2014 [cited by applicant]
US 20140359540A1 · Kelsey et al. · 2014 [cited by applicant]
US 20160170998A1 · Frank et al. · 2016 [cited by applicant]
US 20170083953A1 · Arora · 2017 [cited by applicant]
CN 203606621 · 2014 [cited by applicant]
WO 2007082167 · 2007 [cited by applicant]
Barakonyi, et al., Remote Collaboration Using Augmented Reality Videoconferencing, Citations: p. 1 col. 2 Para 3-p. 2 col. 1 Para 1, p. 3 col. 2 Para 2-Para 3, www.icg.tu-graz.ac.at/publications/pdf/RemoteCollaboration/… [cited by applicant]
Final Office Action, dated May 10, 2016, from corresponding U.S. Appl. No. 14/813,902. [cited by applicant]
Final Office Action, dated May 20, 2016, from corresponding U.S. Appl. No. 14/799,326. [cited by applicant]
Hanna Lee Communications, Inc., Louis Royer's “Show Me the Proof!” Cocktail Competition, Oct. 12, 2014, ShakeStir, pp. 1 and 2 located at http://www.shakestir.com/competitions/ctl/competitiondetails/mid/638/id/8120/loui… [cited by applicant]
Kurillo, et al., Real-Time 3D Avatars for Tele-rehabilitation in Virtual Reality, Citations: p. 4 Para 2, p. 6 Para 2-Para 3, http://www.eecs.berkeley.edu/˜gregorij/MMVR2011.pdf. [cited by applicant]
Notice of Allowance, dated Apr. 7, 2021, from corresponding U.S. Appl. No. 16/833,798. [cited by applicant]
Notice of Allowance, dated Dec. 4, 2019, from corresponding U.S. Appl. No. 16/457,569. [cited by applicant]
Notice of Allowance, dated Feb. 14, 2019, from corresponding U.S. Appl. No. 15/973,286. [cited by applicant]
Notice of Allowance, dated Jan. 9, 2018, from corresponding U.S. Appl. No. 15/367,923. [cited by applicant]
Notice of Allowance, dated Jun. 15, 2023, from corresponding U.S. Appl. No. 17/397,211. [cited by applicant]
Office Action, dated Aug. 20, 2019, from corresponding U.S. Appl. No. 16/457,569. [cited by applicant]
Office Action, dated Dec. 16, 2020, from corresponding U.S. Appl. No. 16/833,798. [cited by applicant]
Office Action, dated Feb. 17, 2017, from corresponding U.S. Appl. No. 14/799,326. [cited by applicant]
Office Action, dated Jan. 11, 2019, from corresponding U.S. Appl. No. 15/263,333. [cited by applicant]
Office Action, dated Mar. 20, 2017, from corresponding U.S. Appl. No. 14/813,902. [cited by applicant]
Office Action, dated Oct. 20, 2022, from corresponding U.S. Appl. No. 17/397,211. [cited by applicant]
Office Action, dated Sep. 22, 2015, from corresponding U.S. Appl. No. 14/813,902. [cited by applicant]
Office Action, dated Sep. 24, 2015, from corresponding U.S. Appl. No. 14/799,326. [cited by applicant]
Office Action, dated Sep. 6, 2018, from corresponding U.S. Appl. No. 15/973,286. [cited by applicant]