IP Library Granted Patent US 10,937,191
Granted Patent B2
US 10,937,191 · App. 16/168,165 · Granted Mar 2, 2021

Predictive simultaneous localization and mapping system using prior user session positional information

Inventors: Vivek Viswanathan Iyer (Austin, TX); Yagiz C. Yildiz (Austin, TX)
Assignee: Dell Products, LP
G06T7/74G01C3/14G01C21/005G01C21/165G02B27/017G06F1/163G06F16/29G06N5/046G06T2207/30244
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,937,191
App. No.
16/168,165
Granted
Mar 2, 2021
Kind
B2
Abstract

An information handling system operating a predictive simultaneous localization and mapping (SLAM) system may comprise a camera mounted to a wearable headset capturing an image of a landmark, a positional sensor mounted to the wearable headset measuring a current positional state of the headset, and a network adapter receiving a stored SLAM frame, including an identification and position of the landmark within a previously captured image. A processor executing code instructions of the predictive SLAM system may generate a current SLAM frame based on the image captured via the camera, determine a Kalman gain associated with an observed location of the landmark based on the current SLAM frame and the stored SLAM frame, and determine a corrected position of the wearable headset based on a previously measured positional state of the headset, the current positional state of the headset, the current SLAM frame, and the Kalman gain.

Claims (53)

1. A head mounted display information handling system operating a predictive simultaneous localization and mapping (SLAM) system comprising:

a camera mounted to a wearable headset capturing a first image of a first stationary landmark at a first location;

a positional sensor mounted to the wearable headset measuring a current positional state of the wearable headset;

a processor generating predicted position coordinates indicating an estimated location of the wearable headset with respect to the first stationary landmark at a future position;

a network adapter receiving a previously stored SLAM frame from a previous location determined to match the predicted position coordinates of the wearable headset with respect to the first stationary landmark at a future position, the previously stored SLAM frame received from a remotely located SLAM frame repository, where the previously stored SLAM frame was generated prior to a current user donning the wearable headset, including a previously captured image at the previous location, an identification of the first stationary landmark within the previously captured image, and position coordinates of the first stationary landmark with respect to the previous location from which a previously captured image was captured to generate the previously stored SLAM frame;

a processor executing code instructions of the predictive SLAM system to:

generate a current SLAM frame of the first stationary landmark based on a second image captured via the camera mounted to the wearable headset;

determine a Kalman gain associated with an observed location of the first stationary landmark based on the current SLAM frame and the previously stored SLAM frame; and

determine corrected position coordinates of the wearable headset with respect to the first stationary landmark based on a previously measured positional state of the headset, a current positional state of the wearable headset, the current SLAM frame, and the Kalman gain.

2. The information handling system of claim 1 , wherein the previous location coordinates from which the previously captured image of the stationary landmark was captured match estimated current predicted coordinates of the wearable headset range and bearing to the first stationary landmark predicted based on the previously measured positional state of the headset.

3. The information handling system of claim 1 further comprising:

the processor identifying a second stationary landmark in the image captured via the camera mounted to the wearable headset; and

the network adapter transmitting the image captured via the camera mounted to the wearable headset and an identification of the second stationary landmark within the image to the remotely located SLAM memory repository for future use with predicted coordinates of the wearable headset with respect to the second stationary landmark.

4. The information handling system of claim 3 , wherein the current SLAM frame associates the captured image with the identification of the second stationary landmark, and a determined position of the second stationary landmark with respect to the wearable headset, further comprising:

the network adapter transmitting the current SLAM frame to the remotely located SLAM memory repository.

5. The information handling system of claim 1 , wherein the camera mounted to the wearable headset is a three-dimensional camera.

6. The information handling system of claim 1 , wherein the positional sensor is a global positioning system (GPS) unit.

7. The information handling system of claim 1 , wherein the corrected position of the wearable headset is determined based on an extended Kalman filter.

8. A method of determining a current positional state of a wearable headset comprising:

receiving, via a network adapter, a current simultaneous localization and mapping (SLAM) frame including an image of a stationary landmark captured via a camera mounted to a remotely located wearable headset, an identification of the stationary landmark within the image, and a position of the stationary landmark with respect to the wearable headset;

predicting an estimated current positional state of the wearable headset based on a previous position state of the wearable headset;

identifying, via a processor, a previously stored SLAM frame generated prior to a current user donning the wearable headset associated within a SLAM memory repository with a previous positional state matching the estimated current positional state of the headset;

determining, via a processor, the previously stored SLAM frame includes an image, identification, and location of the stationary landmark;

determining, via a processor, a Kalman gain associated with the stationary landmark based on the current SLAM frame and the previously stored SLAM frame; and

determining, via a processor, a corrected current position of the wearable headset based on a previously measured positional state of the headset, the current positional state of the headset, the current SLAM frame, and the Kalman gain.

9. The method of claim 8 , wherein the estimated current positional state of the wearable headset includes predicted range and bearing position coordinates to a stationary landmark that is predicted based on the previous positional state of the wearable headset.

10. The method of claim 8 further comprising:

receiving, via the network adapter, the current positional state of the headset measured via a positional sensor mounted to the wearable headset to determine if the received estimated positional state is verified.

11. The method of claim 8 further comprising:

receiving, via the network adapter, the image captured via the camera mounted to the wearable headset and an identification of a second stationary landmark within the image from the wearable headset; and

storing the image and the identification of the second stationary landmark in a SLAM memory repository for future use with predicted estimated current positional state of the wearable headset with respect to the second stationary landmark.

12. The method of claim 11 , wherein the current SLAM frame associates the captured image with the identification of the second stationary landmark, and a determined position of the second stationary landmark with respect to the wearable headset, further comprising:

storing current SLAM frame in the SLAM memory repository.

13. The method of claim 8 , wherein the image captured by the camera mounted to the wearable headset combines a two-dimensional image with a distance between the camera and the first stationary landmark measured by a laser measuring tool mounted on the wearable headset.

14. The method of claim 8 , wherein the positional sensor is an inertial measurement unit.

15. An information handling system operating a predictive simultaneous localization and mapping (SLAM) system comprising:

a network adapter receiving from a wearable headset a first positional state of the headset measured by a positional sensor mounted to the wearable headset;

a processor executing code instructions of the predictive SLAM system to:

predict a future position at a later instance in time of the wearable headset based on the first positional state of the headset;

the processor identifying an image of a stationary landmark stored in a SLAM memory repository and associated with the predicted future position based on a match to a previous position of wearable headset when the image of the stationary landmark stored in the SLAM memory repository was acquired;

the processor associating the image of the stationary landmark with an identification of the later instance in time;

the network adapter receiving, after the later instance in time, a second positional state of the headset measured at the later instance in time by the positional sensor, and a SLAM frame of the stationary landmark generated based on an image of the stationary landmark captured at the later instance in time by a camera mounted to the wearable headset;

the processor executing code instructions of the predictive SLAM system to:

determine a Kalman gain associated with the stationary landmark based on the image of the stationary landmark stored in the SLAM memory repository and the SLAM frame received via the network adapter; and

determine a current state of the wearable headset based on the first positional state of the headset, the second positional state of the headset, the received SLAM frame, and the Kalman gain.

16. The information handling system of claim 15 further comprising:

the processor determining the predicted future position of the wearable headset is within a previously identified area of interest.

17. The information handling system of claim 15 , wherein the image of the stationary landmark stored in the SLAM memory repository is associated with an identification of the stationary landmark within the image, and a determined position of the stationary landmark with respect to the wearable headset.

18. The information handling system of claim 15 further comprising:

the processor executing code instructions of the predictive SLAM system to:

identify the later instance in time by determining when the stationary landmark will become visible to the camera mounted to the wearable headset based on a SLAM frame rate of the wearable headset and the first positional state of the headset.

19. The information handling system of claim 15 , wherein the first positional state of the headset identifies a range, bearing, and velocity of the camera mounted to the wearable headset with respect to the stationary landmark.

20. The information handling system of claim 15 , wherein the positional sensor is an eCompass unit.

Assignments (6)
RELEASE OF SECURITY INTEREST IN PATENTS PREVIOUSLY RECORDED AT REEL/FRAME (053546/0001) Recorded Jun 23, 2022
From: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A., AS NOTES COLLATERAL AGENT
To: DELL MARKETING L.P. (ON BEHALF OF ITSELF AND AS SUCCESSOR-IN-INTEREST TO CREDANT TECHNOLOGIES, INC.); DELL INTERNATIONAL L.L.C.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; DELL MARKETING CORPORATION (SUCCESSOR-IN-INTEREST TO FORCE10 NETWORKS, INC. AND WYSE TECHNOLOGY L.L.C.); EMC IP HOLDING COMPANY LLC
Reel/Frame 071642/0001 →
RELEASE OF SECURITY INTEREST AT REEL 052771 FRAME 0906 Recorded Nov 2, 2021
From: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
To: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
Reel/Frame 058001/0298 →
SECURITY AGREEMENT Recorded May 28, 2020
From: DELL PRODUCTS L.P.; EMC IP HOLDING COMPANY LLC
To: CREDIT SUISSE AG, CAYMAN ISLANDS BRANCH
Reel/Frame 052771/0906 →
SECURITY AGREEMENT Recorded Apr 22, 2020
From: CREDANT TECHNOLOGIES INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 053546/0001 →
SECURITY AGREEMENT Recorded Mar 21, 2019
From: CREDANT TECHNOLOGIES, INC.; DELL INTERNATIONAL L.L.C.; DELL MARKETING L.P.; DELL PRODUCTS L.P.; DELL USA L.P.; EMC CORPORATION; FORCE10 NETWORKS, INC.; WYSE TECHNOLOGY L.L.C.; EMC IP HOLDING COMPANY LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 049452/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 23, 2018
From: IYER, VIVEK VISWANATHAN; YILDIZ, YAGIZ C.
To: DELL PRODUCTS, LP
Reel/Frame 047279/0957 →