IP Library Granted Patent US 10,970,796
Granted Patent B2
US 10,970,796 · App. 16/277,005 · Granted Apr 6, 2021

System and method for hybrid processing of construction site images

Inventors: Michael Sasson (Petah Tikva, IL); Ron Zass (Kiryat Tivon, IL); Shalom Bellaish (Tel Mond, IL); Moshe Nachman (Tel-Aviv, IL)
Assignee: CONSTRU LTD
G06Q50/08G06F16/2379G06F16/583G06K9/00624G06K9/00637G06K9/00671G06K9/623G06K9/6215G06K9/6256G06K9/78G06N20/00G06Q10/06311G06Q40/025G06Q40/08G06Q40/12G06T7/001G06T7/0004G06T7/70G06T2200/24G06T2207/20092G06T2207/30132G06T2207/30242
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,970,796
App. No.
16/277,005
Granted
Apr 6, 2021
Kind
B2
Abstract

Systems and methods for hybrid processing of construction site images are provided. For example, image data captured from a construction site using at least one image sensor may be obtained. The image data may be analyzed to attempt to recognize an object depicted in the image data. In response to a failure to successfully recognize the object, at least part of the image data may be presented to a user, and a feedback related to the object may be received from the user. For example, the attempt to recognize an object may be based on a construction plan associated with the construction site, and the failure to successfully recognize the object may be identified based on a mismatch between the suggested object type from the attempt to recognize the object and types of objects selected from the construction plan based on the location of the object in the image data.

Claims (54)

1. A method for hybrid processing of construction site images, the method comprising:

obtaining image data captured from a construction site using at least one image sensor;

analyzing the image data to attempt to recognize at least one object depicted in the image data;

obtaining a suggested object type from the attempt to recognize the at least one object;

based on a location of the at least one object in the image data, selecting one or more objects in a construction plan associated with the construction site;

obtaining one or more types of the selected one or more objects;

identifying a failure to successfully recognize the at least one object based on a mismatch between the suggested object type and the one or more types of the selected one or more objects; and

in response to the identification of the failure to successfully recognize the at least one object:

presenting at least part of the image data to a user; and

receiving a feedback related to the at least one object from the user.

2. The method of claim 1 , further comprising presenting to the user information based on the construction plan alongside the at least part of the image data.

3. The method of claim 1 , further comprising:

preprocessing the image data using a transformation function to obtain a transformed image comprising at least a convolution of the image data; and

analyzing the transformed image to attempt to recognize the at least one object depicted in the image data.

4. The method of claim 1 , further comprising:

obtaining a suggested object type from the attempt to recognize the at least one object;

selecting one or more objects matching the suggested object type in the construction plan;

obtaining one or more positions specified in the construction plan for the one or more objects matching the suggested object type in the construction plan; and

further basing the identification of the failure to successfully recognize the at least one object based on a mismatch between at least one position of the at least one object in the image data and the one or more positions specified in the construction plan.

5. The method of claim 1 , wherein the attempt to recognize the at least one object is based on a project schedule associated with the construction site, and further comprising presenting to the user information based on the project schedule alongside the at least part of the image data.

6. The method of claim 1 , wherein the identification of the failure to successfully recognize the at least one object is further based on an identification of at least one discrepancy between a recognized at least one object and a project schedule associated with the construction site.

7. The method of claim 1 , wherein the attempt to recognize the at least one object is based on a financial record associated with the construction site, and further comprising presenting to the user information based on the financial record alongside the at least part of the image data.

8. The method of claim 1 , wherein the identification of the failure to successfully recognize the at least one object is further based on an identification of at least one discrepancy between a recognized at least one object and a financial record associated with the construction site.

9. The method of claim 1 , wherein the attempt to recognize the at least one object is based on a progress record associated with the construction site, and further comprising presenting to the user information based on the progress record alongside the at least part of the image data.

10. The method of claim 1 , wherein the identification of the failure to successfully recognize the at least one object is further based on an identification of at least one discrepancy between a recognized at least one object and a progress record associated with the construction site.

11. The method of claim 1 , wherein the identification of the failure to successfully recognize the at least one object is further based on a recognition of the at least one object with a confidence level lower than a selected threshold.

12. The method of claim 1 , wherein the identification of the failure to successfully recognize the at least one object is further based on a successful recognition of a category of the at least one object and a failure to successfully recognize a specific type within the category, and further comprising presenting to the user information associated with the recognized category alongside the at least part of the image data.

13. The method of claim 1 , further comprising analyzing the image data to select the at least part of the image data.

14. The method of claim 1 , further comprising presenting to the user information associated with the construction site alongside the at least part of the image data.

15. The method of claim 1 , further comprising presenting to the user, alongside the at least part of the image data, information associated with the at least one object and determined by analyzing the image data.

16. The method of claim 1 , further comprising presenting to the user information related to a position associated with the at least one object and determined by analyzing the image data alongside the at least part of the image data.

17. The method of claim 1 , further comprising presenting to the user information related to a position associated with at least a portion of the image data alongside the at least part of the image data.

18. The method of claim 1 , further comprising presenting to the user information related to a time associated with at least a portion of the image data alongside the at least part of the image data.

19. A system for hybrid processing of construction site images, the system comprising:

at least one image sensor configured to capture image data from a construction site; and

at least one processor configured to:

analyze the image data to attempt to recognize at least one object depicted in the image data;

obtain a suggested object type from the attempt to recognize the at least one object;

based on a location of the at least one object in the image data, select one or more objects in a construction plan associated with the construction site;

obtain one or more types of the selected one or more objects;

identify a failure to successfully recognize the at least one object based on a mismatch between the suggested object type and the one or more types of the selected one or more objects; and

in response to the identification of the failure to successfully recognize the at least one object:

present at least part of the image data to a user; and

receive a feedback related to the at least one object from the user.

20. A non-transitory computer readable medium storing data and computer implementable instructions for carrying out a method for hybrid processing of construction site images, the method comprising:

obtaining image data captured from a construction site using at least one image sensor;

analyzing the image data to attempt to recognize at least one object depicted in the image data;

obtaining a suggested object type from the attempt to recognize the at least one object;

based on a location of the at least one object in the image data, selecting one or more objects in a construction plan associated with the construction site;

obtaining one or more types of the selected one or more objects;

identifying a failure to successfully recognize the at least one object based on a mismatch between the suggested object type and the one or more types of the selected one or more objects; and

in response to the identification of the failure to successfully recognize the at least one object:

presenting at least part of the image data to a user; and

receiving a feedback related to the at least one object from the user.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 22, 2019
From: SASSON, MICHAEL; ZASS, RON; BELLAISH, SHALOM; NACHMAN, MOSHE
To: CONSTRU LTD
Reel/Frame 049258/0833 →
Continuity (4)
Provisional Application 62631757 · Feb 17, 2018
Provisional Application 62666152 · May 3, 2018
Provisional Application 62791841 · Jan 13, 2019
Related Publication 20190258858A1 · Aug 22, 2019
Cited By (2)
US 12,282,897 US 12,586,133