IP Library Granted Patent US 11,683,462
Granted Patent B2
US 11,683,462 · App. 17/339,703 · Granted Jun 20, 2023

Matching segments of video for virtual display of a space

Inventor: Eugene Herbert Dierks, III (Spokane Valley, WA)
Assignee: Dierks Technology, Inc.
H04N13/156G06T7/74H04N13/122H04N13/221H04N13/282
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,683,462
App. No.
17/339,703
Granted
Jun 20, 2023
Kind
B2
Abstract

Systems, methods, and non-transitory computer-readable medium storing instructions that, when executed, causes a processor to perform operations to display a three-dimensional (3D) space. The methods may include, with an imaging device, capturing a first series of frames as the imaging device travels from a first location to a second location within a space, and capturing a second series of frames as the imaging device travels from the second location to the first location. The method may also include determining a first segment in the first series of frames that matches a second segment in the second series of frames to create a segmentation dataset, generating video clip data based on the segmentation dataset, the video clip data defining a series of video clips, and displaying the series of video clips.

Claims (83)

1. A non-transitory computer-readable medium storing instructions that, when executed, causes a processor to perform operations, comprising:

with an imaging device:

capturing a first series of frames as the imaging device travels from a first location to a second location within a space;

capturing a second series of frames as the imaging device travels from the second location to the first location;

determining a first segment in the first series of frames that matches a second segment in the second series of frames to create a segmentation dataset;

generating video clip data based on the segmentation dataset, the video clip data defining a series of video clips; and

displaying the series of video clips.

2. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed, causes a processor to perform operations comprising:

defining a number of coordinates and a number of angles of capture of a number of frames in the first series of frames and the second series of frames relative to at least a first frame of the number of frames; and

storing the coordinates and angles of capture as mapping data in a database.

3. The non-transitory computer-readable medium of claim 2 , wherein:

the segmentation dataset is created based at least in part on the mapping data, and

generating the series of video clips based on the segmentation dataset includes:

normalizing a distance between the frames in the first series of frames and the second series of frames within the space based on the mapping data;

removing at least one frame of the first series of frames or the second series of frames to normalize a distance traveled along the first segment and the second segment; and

storing the normalized frames as the segmentation dataset.

4. The non-transitory computer-readable medium of claim 2 , wherein defining the coordinates and the angles includes executing a visual simultaneous location and mapping (VSLAM) process.

5. The non-transitory computer-readable medium of claim 2 , wherein:

the number of coordinates include x, y, and z coordinates within the space; and

the number of angles of capture include identification of roll, pitch, and yaw within the space.

6. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed, causes a processor to perform operations comprising:

based at least in part on the video clip data, identifying at least one endpoint of at least two of the video clips; and

defining the endpoint as a decision point within the space at which at least two available directions of movement within the space are presented via a user interface displaying the video clip.

7. The non-transitory computer-readable medium of claim 3 , wherein determining the first segment in the first series of frames that matches the second segment in the second series of frames includes:

estimating which of a number of segments to utilize from within the first series of frames and within the second series of frames based at least in part on a score of best matching frames using at least the distance and angles of capture between the first series of frames and the second series of frames to obtain selected segments;

cropping ends of the selected segments based on a first threshold value; and

collapsing at least one junction point between the segments based on a second threshold to eliminate overlapping segments.

8. The non-transitory computer-readable medium of claim 1 , wherein the imaging device captures the first series of frames and the second series of frames in a single video clip.

9. A client device comprising:

a processor; and

a non-transitory computer-readable media storing instructions that, when executed by the processor, causes the processor to perform operations comprising:

with an imaging device:

capturing a first series of frames as the imaging device travels from a first location to a second location within a space;

capturing a second series of frames as the imaging device travels from the second location to the first location;

defining a number of coordinates and a number of angles of capture of a number of frames in the first series of frames and the second series of frames relative to at least a first frame of the number of frames;

storing the coordinates and angles of capture as mapping data in a database;

determining a first segment in the first series of frames that matches a second segment in the second series of frames based at least in part on the mapping data to create a segmentation dataset;

generating video clip data based on the segmentation dataset, the video clip data defining a series of video clips; and

displaying the series of video clips.

10. The client device of claim 9 , wherein the imaging device is a 360 degree video capture device.

11. The client device of claim 9 , wherein generating the video clip data based on the segmentation dataset includes:

normalizing a distance between the frames in the first series of frames and the second series of frames within the space based on the mapping data;

removing at least one frame of the first series of frames or the second series of frames to normalize a distance traveled along the first segment and the second segment; and

storing the normalized frames as the segmentation dataset.

12. The client device of claim 9 , wherein defining the coordinates and the angles includes:

executing a visual simultaneous location and mapping (VSLAM) process on the captured first series of frames and captured second series of frames to define the coordinates and the angles of capture; and

storing the coordinates and angles of capture as the mapping data.

13. The client device of claim 9 , wherein:

the number of coordinates include x, y, and z coordinates within the space; and

the number of angles of capture include identification of roll, pitch, and yaw within the space.

14. The client device of claim 9 , the operations further comprising:

based at least in part on the video clip data, identifying at least one endpoint of at least two of the video clips; and

defining the endpoint as a decision point within the space at which at least two available directions of movement within the space are presented via a user interface displaying the video clips.

15. The client device of claim 11 , wherein determining the first segment in the first series of frames that matches the second segment in the second series of frames includes:

estimating which of a number of segments to utilize from within the in the first series of frames and the in the second series of frames based at least in part on a score of best matching frames using at least the distance and angles of capture between the first series of frames and the second series of frames to obtain selected segments;

cropping ends of the selected segments based on a first threshold value; and

collapsing at least one junction point between the segments based on a second threshold to eliminate overlapping segments.

16. A method of displaying a three-dimensional (3D) space, comprising:

with an imaging device:

capturing a first series of frames as the imaging device travels from a first location to a second location within a space;

capturing a second series of frames as the imaging device travels from the second location to the first location;

determining a first segment in the first series of frames that matches a second segment in the second series of frames to create a segmentation dataset;

generating video clip data based on the segmentation dataset, the video clip data defining a series of video clips; and

displaying the series of video clips.

17. The method claim 16 , further comprising:

defining a number of coordinates and a number of angles of capture of a number of frames in the first series of frames and the second series of frames relative to at least a first frame of the number of frames based at least in part on execution of a visual simultaneous location and mapping (VSLAM) process; and

storing the coordinates and angles of capture as mapping data in a database,

wherein:

the number of coordinates include x, y, and z coordinates within the space; and

the number of angles of capture include identification of roll, pitch, and yaw within the space.

18. The method claim 17 , wherein:

the segmentation dataset is created based at least in part on the mapping data, and

generating the series of video clips based on the segmentation dataset includes:

normalizing a distance between the frames in the first series of frames and the second series of frames within the space based on the mapping data;

removing at least one frame of the first series of frames or the second series of frames to normalize a distance traveled along the first segment and the second segment; and

storing the normalized frames as the segmentation dataset.

19. The method of claim 16 , further comprising:

based at least in part on the video clip data, identifying at least one endpoint of at least two of the video clips; and

defining the endpoint as a decision point within the space at which at least two available directions of movement within the space are presented via a user interface displaying the video clips.

20. The method of claim 18 , wherein determining the first segment in the first series of frames that matches the second segment in the second series of frames includes:

estimating which of a number of segments to utilize from within the in the first series of frames and the in the second series of frames based at least in part on a score of best matching frames using at least the distance and angles of capture between the first series of frames and the second series of frames to obtain selected segments;

cropping ends of the selected segments based on a first threshold value; and

collapsing at least one junction point between the selected segments based on a second threshold to eliminate overlapping segments.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 4, 2021
From: DIERKS III, EUGENE HERBERT
To: DIERKS TECHNOLOGY, INC.
Reel/Frame 056445/0959 →
Continuity (1)
Related Publication 20220394228A1 · Dec 8, 2022