IP Library Granted Patent US 12,470,892
Granted Patent B2
US 12,470,892 · App. 16/856,720 · Granted Nov 11, 2025

Extended reality localization

Inventor: Michael Chen (Wallingford, PA)
Assignee: Comcast Cable Communications, LLC
H04W4/029G06F3/011G06F16/587G06F16/9537G06V40/172G06V40/70G06V10/17G06V40/103G06V40/16
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,470,892
App. No.
16/856,720
Granted
Nov 11, 2025
Kind
B2
Abstract

Systems, apparatuses, and methods are described for relocalizing a mobile device within an extended reality environment. A subset of a data set, such as a feature pointcloud may be generated based on determining an approximate location of a mobile device. The approximate location may be determined based on data received from one or more sensors in the extended reality environment. Features associated with the approximate location may be included in the subset while other features may be removed. Features extracted from an image taken by the mobile device may be compared with the subset of the data set to relocalize the mobile device.

Claims (65)

1 . A method comprising:

receiving, by a computing device, a request to determine an updated location of a headset in an extended reality (XR) environment;

determining features in an image captured by the headset;

retrieving an image captured by an image sensor associated with the XR environment;

retrieving, via one or more infrared sensors associated with the XR environment, a thermal signature corresponding to the headset;

selecting, based on a distortion in the image captured by the sensor, and based on the thermal signature, a subset of a data set of the XR environment, wherein the distortion is based on an interference within the image captured by the image sensor;

determining the updated location of the headset by comparing the selected subset of the data set with the features determined in the image captured by the headset; and

causing an update to an XR display using the updated location.

2 . The method of claim 1 , wherein selecting the subset of the data set is further based on:

recognizing a body of a user in the image captured by the image sensor; and

not recognizing a face of the user in the image captured by the image sensor.

3 . The method of claim 1 , wherein selecting the subset of the data set is further based on detection of an emission that has interfered with capturing the image by the image sensor.

4 . The method of claim 1 , wherein selecting the subset of the data set is further based on:

causing the headset to emit light; and

detecting the emitted light in the image captured by the image sensor.

5 . The method of claim 1 , wherein selecting the subset of the data set is further based on:

prompting a user of the headset to perform a physical action; and

detecting the physical action in the image captured by the image sensor.

6 . The method of claim 1 , wherein the thermal signature corresponds to heat output by the headset.

7 . The method of claim 1 , wherein selecting the subset of the data set further comprises:

causing the headset to perform processes that cause it to emit heat corresponding to the thermal signature; and

detecting, via the one or more infrared sensors the thermal signature.

8 . The method of claim 1 , wherein the selecting the subset of the data set is further based on:

detecting a motion based on the image captured by the image sensor.

9 . The method of claim 1 , wherein the selecting the subset of the data set further comprises:

determining a direction of a signal received from the headset by a wireless antenna array.

10 . The method of claim 1 , wherein selecting the subset of the data set is further based on:

a triangulation operation utilizing a signal received by an antenna and from the headset.

11 . The method of claim 1 , wherein the determining features in the image captured by the headset further comprises:

receiving information identifying the features.

12 . The method of claim 1 , wherein the XR environment comprises at least one of a virtual reality (VR) environment or an augmented reality (AR) environment.

13 . The method of claim 1 , further comprising:

receiving, from a gateway, information indicating the headset has changed gateways, and wherein the selecting the subset of the data set is further based on the information.

14 . The method of claim 1 , further comprising:

causing a light source to change a color of light being emitted; and

selecting the subset of the data set is based on detecting an effect of the change of the color of the light on the image captured by the image sensor.

15 . The method of claim 1 , wherein the subset of the data set of the XR environment comprises a portion of image data corresponding to a source of the distortion.

16 . The method of claim 1 , wherein the headset is associated with a source of the distortion, and wherein an approximate geographic location of the headset comprises an approximate location of the source of the distortion.

17 . The method of claim 1 , wherein the comparing further comprises:

relocalizing the headset by matching the features determined in the image captured by the headset with the selected subset of the data set.

18 . A method comprising:

receiving, by a computing device, a request to determine an updated location of a mobile device in an extended reality (XR) environment;

receiving, via one or more infrared (IR) sensors, a pattern of thermal emissions;

receiving an image captured by the mobile device;

determining a location of the mobile device based on the pattern of the thermal emissions correlating to a pattern of radio frequency (RF) emissions detected by a gateway device;

removing features from a feature point cloud, based on the location, to generate a subset of a data set;

determining an updated location of the mobile device by comparing the subset of the data set with features in the image; and

causing an update to an XR display using the updated location.

19 . The method of claim 18 , wherein the determining the location of the mobile device is further based on a location of the one or more IR sensors.

20 . A method comprising:

receiving, from a headset, a data set of an extended reality (XR) environment;

determining features in a captured image of the headset;

receiving, from a plurality of gateways, information:

indicating a direction from which first signals of the headset are received by the plurality of gateways; and

identifying a gateway of the plurality of gateways that received second signals comprising a request for an updated location from the headset;

selecting, based on the received information, a subset of the data set;

determining an updated location of the headset by comparing the subset of the data set with the features in the captured image of the headset; and

updating an XR display using the updated location.

21 . The method of claim 20 , wherein selecting the subset of the data set further comprises:

causing the headset to emit light; and

detecting the emitted light in a field of view of a camera.

22 . The method of claim 20 , wherein determining the updated location of the headset is further based on:

performing a plurality of operations to emit a thermal signature;

detecting a pattern of radio frequency (RF) emissions detected by a gateway device; and

correlating the thermal signature with the pattern of RF emissions.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 3, 2020
From: CHEN, MICHAEL
To: COMCAST CABLE COMMUNICATIONS, LLC
Reel/Frame 052827/0056 →
Continuity (1)
Related Publication 20210333863A1 · Oct 28, 2021
References Cited (25)
US 9264479B2 · Mullins · 2016 [cited by examiner]
US 9538332B1 · Mendelson · 2017 [cited by examiner]
US 9851783B2 · Allen · 2017 [cited by applicant]
US 9865091B2 · Giraldi et al. · 2018 [cited by applicant]
US 11089427B1 · Canberk · 2021 [cited by examiner]
US 20100257252A1 · Dougherty et al. · 2010 [cited by applicant]
US 20130293530A1 · Perez · 2013 [cited by examiner]
US 20140324615A1 · Kulkarni · 2014 [cited by examiner]
US 20160191772A1 · Jordan · 2016 [cited by applicant]
US 20160192209A1 · Rumreich · 2016 [cited by examiner]
US 20160217615A1 · Kraver · 2016 [cited by examiner]
US 20170357296A1 · Jordan · 2017 [cited by applicant]
US 20180067187A1 · Oh · 2018 [cited by examiner]
US 20180095542A1 · Mallinson · 2018 [cited by examiner]
US 20180139570A1 · Calvarese · 2018 [cited by examiner]
US 20180348338A1 · Li · 2018 [cited by examiner]
US 20190037134A1 · Merati · 2019 [cited by examiner]
US 20190094919A1 · Jordan · 2019 [cited by applicant]
US 20190122443A1 · Stocker · 2019 [cited by applicant]
US 20190174170A1 · Chen · 2019 [cited by applicant]
US 20200005040A1 · Ur · 2020 [cited by examiner]
US 20200351486A1 · Chapman · 2020 [cited by examiner]
US 20200400327A1 · Thumati · 2020 [cited by examiner]
US 20210110557A1 · Busey · 2021 [cited by examiner]
R. B. Rusu and S. Cousins, “3D is here: Point Cloud Library (PCL),” 2011 IEEE International Conference on Robotics and Automation, Shanghai, 2011, pp. 1-4, doi: 10.1109/ICRA.2011.5980567. [cited by applicant]