IP Library Granted Patent US 10,115,237
Granted Patent B2
US 10,115,237 · App. 15/279,035 · Granted Oct 30, 2018

Virtual reality display of pipe inspection data

Inventors: Justin Starr (Baden, PA); Anthony van Iersel (Auckland, NZ); Timothy Renton (Auckland, NZ); John Lettman (Pittsburgh, PA); Todd Kueny (Tarentum, PA); Foster J. Salotti (Verona, PA)
Assignee: RedZone Robotics, Inc.
G06T19/006G06F3/011H04N5/23238
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,115,237
App. No.
15/279,035
Granted
Oct 30, 2018
Kind
B2
Abstract

One aspect provides a method of projecting pipe data into a virtual reality system, including: obtaining, using a pipe inspection robot, pipe data relating to one or more pipe segments in a pipe network; processing, using a processor, the pipe data to format the pipe data for virtual panoramic display; providing, using the processor, the formatted pipe data to a virtual reality system. Other aspects are described and claimed.

Claims (37)

1. A method of projecting pipe data into a virtual reality (VR) system, comprising:

obtaining, using a pipe inspection robot that traverses through an interior of a pipe, pipe inspection data comprising one or more of laser scan data, sonar scan data, and visual image data, wherein reflection data is sensed and used to determine distance information regarding a distance between one or more of a laser origin and a sonar origin and a sensor;

synchronizing, using a processor, the pipe inspection data with a location within the pipe;

composing, using the processor, a scan image of the location using one or more of the laser scan data and the sonar scan data, the scan image representing a physical shape of the interior of the pipe based on the distance information;

formatting, using a processor, the visual image data and the scan image for virtual panoramic display, the formatting comprising:

projecting the scan image onto a cylinder representing the physical shape of the interior of the pipe to form a scan VR display image; and

projecting the visual image data onto a cylinder representing the physical shape of the interior of the pipe to form a visual VR display image;

providing, using the processor, the scan and visual VR display images to a VR system comprising a head mounted display;

wherein the scan and visual VR display images provide different data source VR images of the location for switching between display of the visual VR display image and the scan VR display image based on user input.

2. The method of claim 1 wherein the pipe inspection data further comprises additional data.

3. The method of claim 2 , wherein the processing comprises combining the additional data with one or more of the scan and visual VR display images.

4. The method of claim 3 , wherein the combining comprises including the additional data as a text or graphic overlay.

5. The method of claim 1 , wherein the providing proceeds in real time.

6. A system, comprising:

a pipe inspection robot that traverses through an interior of a pipe and comprises at least one sensor that obtains pipe inspection data comprising one or more of laser scan data, sonar scan data, and visual image data, wherein reflection data is sensed and used to determine distance information regarding a distance between one or more of a laser origin and a sensor origin and the at least one sensor; and

a computing device including a processor that:

synchronizes the pipe inspection data with a location within the pipe;

composes a scan image of the location using one or more of the laser scan data and the sonar scan data, the scan image representing a physical shape of the interior of the pipe based on the distance information;

formats the visual image data and the scan image for virtual panoramic display, wherein the processor is configured to: project the scan image onto a cylinder representing the physical shape of the interior of the pipe to form a scan virtual reality (VR) display image; and project the visual image data onto a cylinder representing the physical shape of the interior of the pipe to form a visual VR display image;

provides the scan and visual VR display images to a VR system comprising a head mounted display;

wherein the scan and visual VR display images provide different data source VR images of the location for switching between display of the visual VR display image and the scan VR display image based on user input.

7. The system of claim 6 , wherein the at least one sensor comprises a camera having a wide angle lens.

8. The system of claim 6 , wherein the at least one sensor comprises a camera array.

9. The system of claim 6 , wherein the pipe inspection data further comprises additional data.

10. The system of claim 9 , wherein the computing device combines the additional data with one or more of the scan and visual VR display images.

11. The system of claim 10 , wherein the additional data is included as a text or graphic overlay.

12. The system of claim 6 , wherein the computing device is physically attached to the pipe inspection robot.

13. A computer program product, comprising:

a computer readable storage device having code embodied therewith, the code being executable by a processor and comprising:

code that receives pipe inspection data derived from a pipe inspection robot that traverses through an interior of a pipe, the pipe inspection data comprising a scan image and a visual image of a location of the interior of the pipe, the scan image being based on reflection data that is sensed and used to determine distance information regarding a distance between one or more of a laser origin and a sensor origin and a sensor;

wherein the scan image and the visual image are synchronized with a location within the pipe, the scan image representing a physical shape of the interior of the pipe based on the distance information;

code that formats for virtual panoramic display the scan image and the visual image by projection of the scan image and the visual image onto a cylinder representing the physical shape of the interior of the pipe;

code that displays, on a head mounted display device, one or more of the scan image and the visual image as a virtual panoramic display;

code that receives user input to switch the virtual panoramic display; and

code that switches the virtual panoramic display between a visual image panoramic display and a scan image virtual panoramic display in response to the user input.

14. The computer program product of claim 13 , comprising code that modifies the virtual panoramic display to update a view perspective to a new area of the virtual panoramic display.

15. The computer program product of claim 13 , comprising code that modifies the virtual panoramic display to add additional data in the form of text or a graphic.

Assignments (6)
RELEASE OF SECURITY INTEREST Recorded Dec 20, 2022
From: MONROE CAPITAL MANAGEMENT ADVISORS, LLC
To: REDZONE ROBOTICS, INC., A DELAWARE CORPORATION
Reel/Frame 062160/0326 →
SECURITY INTEREST Recorded Dec 20, 2022
From: REDZONE ROBOTICS, INC.; RZR HOLDCO, INC.; RZR BUYER SUB, INC.
To: FIRST COMMONWEALTH BANK
Reel/Frame 062160/0976 →
SECURITY INTEREST Recorded Sep 17, 2019
From: REDZONE ROBOTICS, INC.
To: MONROE CAPITAL MANAGEMENT ADVISORS, LLC, AS ADMINISTRATIVE AGENT
Reel/Frame 050407/0158 →
SECURITY INTEREST Recorded Feb 25, 2019
From: REDZONE ROBOTICS, INC.
To: MONROE CAPITAL MANAGEMENT ADVISORS, LLC, AS ADMINISTRATIVE AGENT
Reel/Frame 048428/0963 →
SECURITY INTEREST Recorded Jun 4, 2018
From: REDZONE ROBOTICS, INC.
To: MONROE CAPITAL MANAGEMENT ADVISORS, LLC, AS ADMINISTRATIVE AGENT
Reel/Frame 045982/0558 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 28, 2016
From: STARR, JUSTIN; VAN IERSEL, ANTONY; RENTON, TIMOTHY; LETTMAN, JOHN; KUENY, TODD; SALOTTI, FOSTER J.
To: REDZONE ROBOTICS, INC.
Reel/Frame 040172/0315 →
Continuity (1)
Related Publication 20180089896A1 · Mar 29, 2018