IP Library Granted Patent US 11,526,992
Granted Patent B2
US 11,526,992 · App. 16/673,511 · Granted Dec 13, 2022

Imagery-based construction progress tracking

Inventor: Philip Garcia Lorenzo (Sacramento, CA)
G06T7/11G06T11/203G06V10/44G06V20/46H04N5/23238
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,526,992
App. No.
16/673,511
Granted
Dec 13, 2022
Kind
B2
Abstract

A method is provided. The method includes one or more of receiving, by an image processing device, one or more images from an image capture device. The one or more images are each associated with metadata that includes a common direction. For each of the one or more images, the method further includes adding one or more pairs of parallel lines, converting each of the one or more pairs of parallel lines into intersection coordinates with 2D drawing elements, and calculating construction progress from the intersection coordinates. The 2D drawing includes a 2D floor plan or a 2D elevation plan, and each pair of parallel lines designates one of the start or end of construction during a current period of time.

Claims (70)

1. A method comprising:

receiving, by an image processing device, one or more images from an image capture device, the one or more images each associated with metadata comprising a common direction, and for each of the one or more images:

adding one or more pairs of parallel lines, each pair of parallel lines designating one of the start or end of construction during a current period of time;

converting each of the one or more pairs of parallel lines into intersection coordinates with 2D drawing elements, the 2D drawing comprising a 2D floor plan or a 2D elevation plan; and

calculating construction progress from the intersection coordinates,

wherein each of the one or more images comprises one of a frame extracted from a video, a 360 degree photo, or a non-360 degree photo, wherein the one or more images are captured at predetermined locations associated with a building, wherein the metadata comprises a common direction reference for all captured images.

2. The method of claim 1 , wherein each pair of parallel lines is one of oriented vertically and reflecting a pair of yaw values relative to the common direction or oriented horizontally and reflecting a pair of pitch values relative to a common unpitched value.

3. The method of claim 2 , wherein prior to adding the one or more pairs of parallel lines to the one or more images, the method further comprising:

identifying construction objects of interest to be tracked within each of the one or more images; and

segmenting the construction objects of interest, comprising joining line segments enclosing each of the objects of interest.

4. The method of claim 3 , the method further comprising:

updating a total construction progress from the calculated construction progress, wherein the total and calculated construction progress reflects one of a percent complete or a quantity;

comparing one or more of the calculated construction progress or the total construction progress to a schedule; and

providing a notification in response to one or more of the calculated construction progress or the total construction progress does not match the schedule, wherein the notification comprises a difference between the one or more of the calculated construction progress or the total construction progress and the schedule.

5. The method of claim 3 , wherein converting each of the one or more pairs of parallel lines into intersection coordinates with 2D drawing elements comprising:

mapping line segments in the 2D drawing that corresponds to elements of the building;

converting yaw or pitch degree lines added to each of the one or more images into rays in the 2D drawing, wherein each of the rays extends from a position where one of the images was captured to a location on an element that corresponds to a yaw or pitch line; and

determining intersection coordinates where each ray intersects a first line segment along the ray.

6. The method of claim 5 , wherein calculating construction progress from the intersection coordinates comprising one of:

calculating a linear distance between intersection coordinates, wherein the linear distance comprises a distance along building elements between a pair of intersection coordinates; or

calculating a number or portion of a specified element between the intersection coordinates; and

consulting one or more different images and recalculating construction progress for the one or more different images to establish a higher construction progress confidence level.

7. An image processing device, comprising:

a memory, comprising:

one or more applications;

a 2D drawing comprising building locations of a building, the 2D drawing comprising a 2D floor plan or a 2D elevation plan; and

one or more images of the building locations, each image associated with metadata comprising a common direction,

wherein each of the one or more images comprises one of a frame extracted from a video, a 360 degree photo, or a non-360 degree photo, wherein the one or more images are captured at predetermined locations associated with the building, wherein the metadata comprises a common direction reference for all captured images; and

a processor, coupled to the memory, configured to execute the one or more applications to:

receive the one or more images from an image capture device, and for each of the one or more images:

add one or more pairs of parallel lines, each pair of parallel lines designates one of the start or end of construction during a current period of time;

convert each of the one or more pairs of parallel lines into intersection coordinates with 2D drawing elements; and

calculate construction progress from the intersection coordinates.

8. The system of claim 7 , wherein each pair of parallel lines is one of oriented vertically and reflects a pair of yaw values relative to the common direction or oriented horizontally and reflects a pair of pitch values relative to a common unpitched value.

9. The system of claim 8 , wherein prior to the processor adds the one or more pairs of parallel lines to the one or more images, the processor is configured to

identify construction objects of interest to be tracked within each of the one or more images; and

segment the construction objects of interest, comprising joined line segments that enclose each of the objects of interest.

10. The system of claim 9 , wherein the processor is further configured to:

update a total construction progress from the calculated construction progress, wherein the total and calculated construction progress reflects one of a percent complete or a quantity;

compare one or more of the calculated construction progress or the total construction progress to a schedule; and

provide a notification in response to one or more of the calculated construction progress or the total construction progress does not match the schedule, wherein the notification comprises a difference between the one or more of the calculated construction progress or the total construction progress and the schedule.

11. The system of claim 9 , wherein the processor converts the one or more pairs of parallel lines into intersection coordinates with 2D drawing elements comprises the processor is further configured to:

map line segments in the 2D drawing that correspond to elements of the building;

convert yaw or pitch degree lines added to each of the one or more images into rays in the 2D drawing, wherein each of the rays extends from a position where one of the images was captured to a location on an element that corresponds to a yaw or pitch line; and

determine intersection coordinates where each ray intersects a first line segment along the ray.

12. The system of claim 11 , wherein the processor calculates construction progress from the intersection coordinates comprises the processor is further configured to one of:

calculate a linear distance between intersection coordinates, wherein the linear distance comprises a distance along building elements between a pair of intersection coordinates; or

calculate a number or portion of a specified element between the intersection coordinates; and

consult one or more different images to recalculate construction progress for the one or more different images to establish a higher construction progress confidence level.

13. A non-transitory computer readable storage medium configured to store instructions that when executed cause a processor to perform:

receiving, by an image processing device, one or more images from an image capture device, the one or more images each associated with metadata comprising a common direction, and for each of the one or more images:

adding one or more pairs of parallel lines, each pair of parallel lines designating one of the start or end of construction during a current period of time;

converting each of the one or more pairs of parallel lines into intersection coordinates with 2D drawing elements, the 2D drawing comprising a 2D floor plan or a 2D elevation plan; and

calculating construction progress from the intersection coordinates,

wherein each of the one or more images comprises one of a frame extracted from a video, a 360 degree photo, or a non-360 degree photo, wherein the one or more images are captured at predetermined locations associated with a building, wherein the metadata comprises a common direction reference for all captured images.

14. The non-transitory computer readable storage medium of claim 13 , wherein each pair of parallel lines is one of oriented vertically and reflecting a pair of yaw values relative to the common direction or oriented horizontally and reflecting a pair of pitch values relative to a common unpitched value, wherein prior to adding the one or more pairs of parallel lines to the one or more images, the instructions are further configured to cause the processor to:

identify construction objects of interest to be tracked within each of the one or more images; and

segment the construction objects of interest, comprising joining line segments enclosing each of the objects of interest.

15. The non-transitory computer readable storage medium of claim 14 , wherein the instructions are further configured to cause the processor to:

update a total construction progress from the calculated construction progress, wherein the total and calculated construction progress reflects one of a percent complete or a quantity;

compare one or more of the calculated construction progress or the total construction progress to a schedule; and

provide a notification in response to one or more of the calculated construction progress or the total construction progress does not match the schedule, wherein the notification comprises a difference between the one or more of the calculated construction progress or the total construction progress and the schedule.

16. The non-transitory computer readable storage medium of claim 14 , wherein converting each of the one or more pairs of parallel lines into intersection coordinates with 2D drawing elements comprising:

mapping line segments in the 2D drawing that corresponds to elements of the building;

converting yaw or pitch degree lines added to each of the one or more images into rays in the 2D drawing, wherein each of the rays extends from a position where one of the images was captured to a location on an element that corresponds to a yaw or pitch line; and

determining intersection coordinates where each ray intersects a first line segment along the ray.

17. The non-transitory computer readable storage medium of claim 16 , wherein calculating construction progress from the intersection coordinates comprising one of:

calculating a linear distance between intersection coordinates, wherein the linear distance comprises a distance along building elements between a pair of intersection coordinates; or

calculating a number or portion of a specified element between the intersection coordinates; and

consulting one or more different images and recalculating construction progress for the one or more different images to establish a higher construction progress confidence level.

Assignments (5)
SECURITY INTEREST Recorded Sep 3, 2025
From: DRONEDEPLOY, INC.; STRUCTIONSITE, INC.
To: HERCULES CAPITAL, INC., AS AGENT
Reel/Frame 072147/0547 →
SECURITY INTEREST Recorded Nov 10, 2024
From: DRONEDEPLOY, INC.
To: HERCULES CAPITAL, INC., AS AGENT
Reel/Frame 069196/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 1, 2024
From: STRUCTIONSITE, INC.
To: DRONEDEPLOY, INC.
Reel/Frame 066968/0227 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 23, 2021
From: LORENZO, PHILIP GARCIA
To: STRUCTIONSITE INC.
Reel/Frame 056321/0690 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 4, 2019
From: LORENZO, PHILIP GARCIA
To: STRUCTIONSITE INC.
Reel/Frame 050909/0449 →
Continuity (3)
Continuation 16509340 · Jul 11, 2019
Provisional Application 62697472 · Jul 13, 2018
Related Publication 20200065971A1 · Feb 27, 2020