IP Library Granted Patent US 11,769,238
Granted Patent B2
US 11,769,238 · App. 17/925,958 · Granted Sep 26, 2023

Systems and methods for creating and/or analyzing three-dimensional models of infrastructure assets

Inventors: Benjamin T. Schmidt (Pittsburgh, PA); Miguel Dickson (Pittsburgh, PA); Christoph Mertz (Pittsburgh, PA); Anshul Goyal (Pittsburgh, PA)
Assignee: RoadBotics, Inc.
G06T7/0002G06T7/20G06T7/90G06T17/00G06V20/588G06T2207/10016G06T2207/10024G06T2207/10028G06T2207/30184G06T2207/30256G06T2210/56
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,769,238
App. No.
17/925,958
Granted
Sep 26, 2023
Kind
B2
Abstract

Systems and methods for detecting, geolocating, assessing, and/or inventorying infrastructure assets. In some embodiments, a plurality of images captured by a moving camera may be used to generate a point cloud. A plurality of points corresponding to a pavement surface may be identified from the point cloud. The plurality of points may be used to generate at least one synthetic image of the pavement surface, the at least one synthetic image having at least one selected camera pose. The at least one synthetic image may be used to assess at least one condition of the pavement surface.

Claims (68)

1. A computer-implemented method comprising acts of:

using a plurality of images captured by a moving camera to generate a point cloud;

identifying, from the point cloud, a plurality of points corresponding to a pavement surface of a road segment;

using the plurality of points to generate a plurality of synthetic images of the pavement surface, wherein:

the plurality of synthetic images have, respectively, a plurality of selected camera poses distributed along the road segment;

the plurality of selected camera poses have a same height and a same orientation relative to the road segment;

the pavement surface comprises a plurality of non-overlapping intervals; and

each synthetic image of the plurality of synthetic images shows a respective interval of the plurality of non-overlapping intervals; and

using the plurality of synthetic images to assess at least one condition of the pavement surface of the road segment.

2. The method of claim 1 , wherein:

identifying a plurality of points corresponding to a pavement surface comprises identifying a plurality of points having surface normals that are within a selected threshold of vertical.

3. The method of claim 1 , wherein:

identifying a plurality of points corresponding to a pavement surface comprises identifying a plurality of points having pixel colors that match one or more selected pavement colors.

4. The method of claim 1 , wherein:

the point cloud is georeferenced based on geospatial metadata associated with the plurality of images; and

the plurality of selected camera poses comprise, respectively, a plurality of selected georeferenced positions.

5. The method of claim 1 , further comprising acts of:

estimating a motion of the camera; and

sampling, based on the estimated motion of the camera, from a video stream captured by the camera, the plurality of images used to generate the point cloud.

6. The method of claim 5 , wherein:

the video stream is sampled at a first frequency in response to determining that the estimated motion of the camera includes rotational motion; and

the video stream is sampled at a second frequency in response to determining that the estimated motion of the camera does not include rotational motion, wherein the second frequency is lower than the first frequency.

7. A system comprising:

at least one processor; and

at least one computer-readable storage medium having stored thereon instructions which, when executed, program the at least one processor to:

use a plurality of images captured by a moving camera to generate a point cloud;

identify, from the point cloud, a plurality of points corresponding to a pavement surface of a road segment;

use the plurality of points to generate a plurality of synthetic images of the pavement surface, wherein:

the plurality of synthetic images have, respectively, a plurality of selected camera poses distributed along the road segment;

the plurality of selected camera poses have a same height and a same orientation relative to the road segment;

the pavement surface comprises a plurality of non-overlapping intervals; and

each synthetic image of the plurality of synthetic images shows a respective interval of the plurality of non-overlapping intervals; and

use the plurality of synthetic images to assess at least one condition of the pavement surface of the road segment.

8. The system of claim 7 , wherein the at least one processor is programmed to:

identify a plurality of points corresponding to a pavement surface at least in part by identifying a plurality of points having surface normals that are within a selected threshold of vertical.

9. The system of claim 7 , wherein the at least one processor is programmed to:

identify a plurality of points corresponding to a pavement surface at least in part by identifying a plurality of points having pixel colors that match one or more selected pavement colors.

10. The system of claim 7 , wherein:

the point cloud is georeferenced based on geospatial metadata associated with the plurality of images; and

the plurality of selected camera poses comprise, respectively, a plurality of selected georeferenced positions.

11. The system of claim 7 , wherein the at least one processor is further programmed to:

estimate a motion of the camera; and

sample, based on the estimated motion of the camera, from a video stream captured by the camera, the plurality of images used to generate the point cloud.

12. The system of claim 11 , wherein the at least one processor is programmed to:

sample the video stream at a first frequency in response to determining that the estimated motion of the camera includes rotational motion; and

sample the video stream at a second frequency in response to determining that the estimated motion of the camera does not include rotational motion, wherein the second frequency is lower than the first frequency.

13. At least one non-transitory computer-readable medium having stored thereon instructions which, when executed, program at least one processor to perform a method comprising acts of:

using a plurality of images captured by a moving camera to generate a point cloud;

identifying, from the point cloud, a plurality of points corresponding to a pavement surface of a road segment;

using the plurality of points to generate a plurality of synthetic images of the pavement surface, wherein:

the plurality of synthetic images have, respectively, a plurality of selected camera poses distributed along the road segment;

the plurality of selected camera poses have a same height and a same orientation relative to the road segment;

the pavement surface comprises a plurality of non-overlapping intervals; and

each synthetic image of the plurality of synthetic images shows a respective interval of the plurality of non-overlapping intervals; and

using the plurality of synthetic images to assess at least one condition of the pavement surface of the road segment.

14. The at least one non-transitory computer-readable medium of claim 13 , wherein:

identifying a plurality of points corresponding to a pavement surface comprises identifying a plurality of points having surface normals that are within a selected threshold of vertical.

15. The at least one non-transitory computer-readable medium of claim 13 , wherein:

identifying a plurality of points corresponding to a pavement surface comprises identifying a plurality of points having pixel colors that match one or more selected pavement colors.

16. The at least one non-transitory computer-readable medium of claim 13 , wherein:

the point cloud is georeferenced based on geospatial metadata associated with the plurality of images; and

the plurality of selected camera poses comprise, respectively, a plurality of selected georeferenced positions.

17. The at least one non-transitory computer-readable medium of claim 13 , wherein the method further comprises acts of:

estimating a motion of the camera; and

sampling, based on the estimated motion of the camera, from a video stream captured by the camera, the plurality of images used to generate the point cloud.

18. The at least one non-transitory computer-readable medium of claim 17 , wherein:

the video stream is sampled at a first frequency in response to determining that the estimated motion of the camera includes rotational motion; and

the video stream is sampled at a second frequency in response to determining that the estimated motion of the camera does not include rotational motion, wherein the second frequency is lower than the first frequency.

Assignments (2)
CORRECTIVE ASSIGNMENT TO CORRECT THE THE 4TH INVENTOR'S NAME PREVIOUSLY RECORDED AT REEL: 063060 FRAME: 0001. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Aug 4, 2023
From: SCHMIDT, BENJAMIN T.; DICKSON, MIGUEL; MERTZ, CHRISTOPH; GOYAL, ANSHUL
To: ROADBOTICS, INC.
Reel/Frame 064491/0894 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 22, 2023
From: SCHMIDT, BENJAMIN T.; DICKSON, MIGUEL; MERTZ, CHRISTOPH; GOYAL, ASHUN
To: ROADBOTICS, INC.
Reel/Frame 063060/0001 →
Continuity (2)
Provisional Application 63026602 · May 18, 2020
Related Publication 20230196535A1 · Jun 22, 2023
Cited By (2)
US 12,249,057 US 12,475,375