IP Library › Granted Patent US 11,605,151
Granted Patent B2
US 11,605,151 · App. 17/190,268 · Granted Mar 14, 2023

Vehicle undercarriage imaging

Inventors: Stefan Johannes Josef Holzer (San Mateo, CA); Krunal Ketan Chande (San Francisco, CA); Matteo Munaro (San Francisco, CA); Pavel Hanchar (Minsk, BY); Aidas Liaudanskas (San Francisco, CA); Wook Yeon Hwang (San Francisco, CA); Blake McConnell (San Francisco, CA); Johan Nordin (San Francisco, CA); Milos Vlaski (San Francisco, CA); Martin Markus Hubert Wawro (Dortmund, DE); Nick Stetco (San Francisco, CA); Martin Saelzle (San Francisco, CA)
Assignee: Fyusion, Inc.
G06T3/4038G06T7/0004G06T7/50G06T2207/10028G06T2207/20221G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,605,151
App. No.
17/190,268
Granted
Mar 14, 2023
Kind
B2
Abstract

Images of an undercarriage of a vehicle may be captured via one or more cameras. A point cloud may be determined based on the images. The point cloud may includes points positioned in a virtual three-dimensional space. A stitched image may be determined based on the point cloud by projecting the point cloud onto a virtual camera view. The stitched image may be stored on a storage device.

Claims (29)

1. A method comprising:

capturing a plurality of images of an undercarriage of a vehicle, the images captured via one or more cameras, each camera being arranged in a respective fixed position, the images being captured as the vehicle moves over the one or more cameras;

determining a designated point cloud based on the plurality of images via a processor, the designated point cloud including a plurality of points positioned in a virtual three-dimensional space;

determining a designated stitched image based on the designated point cloud via the processor, wherein the stitched image is determined by projecting the designated point cloud onto a virtual camera view, wherein determining the stitched image comprises identifying a plurality of correspondence points, each of the correspondence points indicating a visual feature included in two or more of the plurality of images, wherein determining the stitched image further comprises identifying a plurality of point correspondence transformations, each of the point correspondence transformations identifying a respective spatial relationship between a first location of a designated one of the correspondence points in a first one of the images and a second location of the designated correspondence point in a second one of the images; and

storing the designated stitched image on a storage device.

2. The method recited in claim 1 , wherein the virtual camera view is a two-dimensional plane in the virtual three-dimensional space.

3. The method recited in claim 1 , wherein the virtual camera view is a point in the virtual three-dimensional space.

4. The method recited in claim 1 , wherein determining the designated point cloud comprises determining a plurality of point clouds.

5. The method recited in claim 4 , wherein the one or more cameras includes a plurality of cameras, and wherein each of the point clouds corresponds with a respective one of the cameras.

6. The method recited in claim 4 , wherein each of the point clouds corresponds with a respective one of the images.

7. The method recited in claim 1 , wherein the designated point cloud is determined based at least in part on depth information captured from one or more depth sensors.

8. The method recited in claim 1 , wherein determining the stitched image further comprises determining an image correspondence providing a mathematical transformation between the first and second images in a two or three dimensional coordinate space based on the plurality of correspondence transformations.

9. The method recited in claim 8 , wherein the image correspondence is a numerical average of one or more of the point correspondence transformations, the numerical average determined based on a cost function.

10. The method recited in claim 1 , wherein the designated stitched image is one of a plurality of stitched images determined based on the designated point cloud.

11. The method recited in claim 1 , the method further comprising generating a multi-view interactive digital media representation that includes the plurality of stitched images, the multi-view interactive digital media representation being navigable in one or more dimensions.

12. The method recited in claim 11 , wherein the multi-view interactive digital media representation being navigable in three dimensions.

13. A system comprising:

one or more cameras configured to capture a plurality of images of an undercarriage of a vehicle, each camera being arranged in a respective fixed position, the images being captured as the vehicle moves over the one or more cameras;

a processor configured to determine a designated point cloud based on the plurality of images and determine a designated stitched image based on the designated point cloud, the designated point cloud including a plurality of points positioned in a virtual three-dimensional space, wherein the stitched image is determined by projecting the designated point cloud onto a virtual camera view, wherein determining the stitched image comprises identifying a plurality of correspondence points, each of the correspondence points indicating a visual feature included in two or more of the plurality of images, wherein determining the stitched image further comprises identifying a plurality of point correspondence transformations, each of the point correspondence transformations identifying a respective spatial relationship between a first location of a designated one of the correspondence points in a first one of the images and a second location of the designated correspondence point in a second one of the images; and

a storage device configured to store the designated stitched image.

14. The system recited in claim 13 , wherein determining the stitched image further comprises determining an image correspondence providing a mathematical transformation between the first and second images in a two or three dimensional coordinate space based on the plurality of correspondence transformations, and wherein the image correspondence is a numerical average of one or more of the point correspondence transformations, the numerical average determined based on a cost function.

15. The system recited in claim 13 , wherein the processor is further operable to generate a multi-view interactive digital media representation that includes the plurality of stitched images, the multi-view interactive digital media representation being navigable in one or more dimensions.

16. The system recited in claim 13 , wherein determining the designated point cloud comprises determining a plurality of point clouds, and wherein the one or more cameras includes a plurality of cameras, and wherein each of the point clouds corresponds with a respective one of the cameras.

17. The method recited in claim 1 , wherein the virtual camera view is a two-dimensional plane of a point in the virtual three-dimensional space.

18. One or more non-transitory computer readable media having instructions stored thereon for performing a method, the method comprising:

capturing a plurality of images of an undercarriage of a vehicle, the images captured via one or more cameras, each camera being arranged in a respective fixed position, the images being captured as the vehicle moves over the one or more cameras;

determining a designated point cloud based on the plurality of images via a processor, the designated point cloud including a plurality of points positioned in a virtual three-dimensional space;

determining a designated stitched image based on the designated point cloud via the processor, wherein the stitched image is determined by projecting the designated point cloud onto a virtual camera view, wherein determining the stitched image comprises identifying a plurality of correspondence points, each of the correspondence points indicating a visual feature included in two or more of the plurality of images, wherein determining the stitched image further comprises identifying a plurality of point correspondence transformations, each of the point correspondence transformations identifying a respective spatial relationship between a first location of a designated one of the correspondence points in a first one of the images and a second location of the designated correspondence point in a second one of the images; and

storing the designated stitched image on a storage device.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2021
From: HOLZER, STEFAN JOHANNES JOSEF; CHANDE, KRUNAL KETAN; MUNARO, MATTEO; HANCHAR, PAVEL; LIAUDANSKAS, AIDAS; HWANG, WOOK YEON; MCCONNELL, BLAKE; NORDIN, JOHAN; VLASKI, MILOS; WAWRO, MARTIN MARKUS HUBERT; STETCO, NICK; SAELZLE, MARTIN
To: FYUSION, INC.
Reel/Frame 055538/0468 →
Continuity (1)
Related Publication 20220284544A1 · Sep 8, 2022
Cited By (1)
US 12,488,483