IP Library › Granted Patent US 12,243,178
Granted Patent B2
US 12,243,178 · App. 18/496,407 · Granted Mar 4, 2025

Cross reality system with localization service and shared location-based content

Inventors: Timothy Dean Caswell (Leander, TX); Konrad Piascik (Oakville, CA); Leonid Zolotarev (Weston, FL); Mark Ashley Rushton (Fort Lauderdale, FL)
Assignee: Magic Leap, Inc.
G06T19/006G06T15/005G06T15/20
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,243,178
App. No.
18/496,407
Granted
Mar 4, 2025
Kind
B2
Abstract

A cross reality system enables any of multiple devices to efficiently render shared location-based content. The cross reality system may include a cloud-based service that responds to requests from devices to localize with respect to a stored map. The service may return to the device information that localizes the device with respect to the stored map. In conjunction with localization information, the service may provide information about locations in the physical world proximate the device for which virtual content has been provided. Based on information received from the service, the device may render, or stop rendering, virtual content to each of multiple users based on the user's location and specified locations for the virtual content.

Claims (71)

1. A networked resource within a distributed computing environment for providing shared location-based content to a plurality of portable electronic devices capable of rendering virtual content in a 3D environment, the networked resource comprising:

one or more processors:

at least one computer readable medium comprising:

a plurality of stored maps of the 3D environment;

a plurality of data structures, wherein:

each data structure of the plurality of data structures comprises:

information associating the data structure with a location in the plurality of stored maps,

a link to virtual content for rendering in a respective region in the 3D environment, and

display properties for the respective region on the plurality of portable electronic devices; and

the display properties comprise one or more of: dimensions of the respective region, offset of the respective region from the location in the plurality of stored maps, spatial orientation of the respective region, and behavior of virtual content rendered in the respective region; and

computer executable instructions that, when executed by at least one processor of the one or more processors:

provide a copy of a data structure of the plurality of data structures to a portable electronic device of the plurality of portable electronic devices based on a coordinate frame of the portable electronic device with respect to the plurality of data structures.

2. The networked resource of claim 1 , wherein:

the computer executable instructions, when executed by the at least one processor, an authentication service that determines access rights of the portable electronic device; and

the computer executable instructions that provide the copy of the data structure to the portable electronic device determine, in part, whether to send the copy of the data structure based on the access rights of the portable electronic device and access attributes associated with the copy of the data structure.

3. The networked resource of claim 1 , wherein:

each data structure of the plurality of data structures further comprises a public attribute; and

the computer executable instructions that provide the copy of the data structure to the portable electronic device determine, in part, whether to send the copy of the data structure based on the public attribute of the copy of the data structure.

4. The networked resource of claim 1 , wherein:

for a portion of the plurality of data structures, the link to virtual content comprises a link to an application supplying virtual content.

5. The networked resource of claim 1 , wherein:

the respective region is a volume within which the virtual content linked to the data structure is displayed.

6. The networked resource of claim 1 , wherein:

the behavior of the virtual content comprises the behavior of the virtual content with respect to physical surfaces.

7. The networked resource of claim 1 , wherein:

the behavior of the virtual content rendered in the respective region comprises the behavior of the virtual content with respect to the coordinate frame of the portable electronic device, and the behavior of the virtual content with respect to a direction in which the portable electronic device is facing.

8. A method of operating a portable electronic device to render virtual content in a 3D environment, the method comprising, with one or more processors:

sending over a network to a service, information indicative of a location in the 3D environment;

obtaining from the service one or more data structures, each representing a respective region in the 3D environment and virtual content for display in the respective region;

rendering, in the respective region of the one or more data structures, the virtual content represented in the one or more data structures;

detecting that the portable electronic device has moved away from a region represented by a data structure of the one or more data structures; and

based on the detecting, deleting virtual content represented in the data structure.

9. The method of claim 8 , wherein:

rendering the virtual content in the respective region comprises creating the respective region having parameters set based on the data structure representing the respective region.

10. The method of claim 8 , wherein the virtual content is represented in at least one of the one or more data structures as an indicator of a location of the virtual content on a network.

11. The method of claim 8 , wherein:

rendering the virtual content comprises executing an application on the portable electronic device that generates the virtual content.

12. The method of claim 11 , wherein rendering the virtual content further comprises:

determining whether the application is currently installed on the portable electronic device; and

based on determining that the application is not currently installed, downloading the application to the portable electronic device.

13. The method of claim 8 , wherein:

each of the one or more data structures comprises display properties for the respective region on the portable electronic device; and

the display properties comprise one or more of: dimensions of the respective region, offset of the respective region from the location in the 3D environment, spatial orientation of the respective region, and behavior of virtual content rendered in the respective region.

14. The method of claim 8 , wherein:

the one or more data structures comprise a first set of data structures;

the first set of data structures is received at a first time; and

the method further comprises:

storing rendering information associated with a first data structure in the first set of data structures;

receiving at a second time, after the first time, a second set of data structures; and

based on a determination that the first data structure is not contained in the second set of data structures, deleting the rendering information associated with the first data structure.

15. An electronic device configured to operate within a cross reality system, the electronic device comprising:

one or more sensors configured to capture information about a three-dimensional (3D) environment, the captured information comprising a plurality of images;

at least one processor; and

at least one computer readable medium storing computer executable instructions that, when executed on a processor of the at least one processor:

maintain a local coordinate frame;

manage respective regions associated with one or more applications such that virtual content generated by an application of the one or more applications is rendered within a respective region;

receive, at a first time, from a service:

a first data structure representing respective virtual content and a respective region in the 3D environment for rendering the virtual content;

associate a respective region with the first data structure such that the respective virtual content is rendered within the respective region;

receive, at a second time after the first time, from the service:

a second set of data structures; and

based on a determination that the first data structure is not contained in the second set of data structures, delete the rendering information associated with the first data structure.

16. The electronic device of claim 15 , wherein the computer executable instructions further comprise computer executable instructions for:

obtaining, based on information in the first data structure, the respective virtual content; and

rendering, within the respective region, the obtained respective virtual content.

17. The electronic device of claim 16 , wherein obtaining the respective virtual content, based on information in the first data structure, comprises accessing the respective virtual content over a network based on an indicator of a location of virtual content in the first data structure.

18. The electronic device of claim 17 , wherein obtaining the respective virtual content, based on information in the first data structure, comprises downloading over the network an application that generates the respective virtual content based on the indicator of the location of virtual content in the first data structure.

19. The electronic device of claim 15 , wherein the computer executable instructions further comprise instructions for:

detecting that the electronic device has moved away from a region represented by the first data structure; and

based on the detecting, deleting the respective region associated with the first data structure.

20. The electronic device of claim 16 , wherein rendering, within the respective region, the obtained virtual content further comprises determining a set of coordinates in the 3D environment at which to render the virtual content using a coordinate system of the electronic device.

Assignments (3)
SECURITY INTEREST Recorded Oct 31, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073422/0549 →
SECURITY INTEREST Recorded Oct 20, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073031/0206 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 29, 2023
From: CASWELL, TIMOTHY DEAN; PIASCIK, KONRAD; ZOLOTAREV, LEONID; RUSHTON, MARK ASHLEY
To: MAGIC LEAP, INC.
Reel/Frame 065983/0992 →
Continuity (4)
Continuation 17824839 · May 25, 2022
Continuation 17095551 · Nov 11, 2020
Provisional Application 62934485 · Nov 12, 2019
Related Publication 20240054741A1 · Feb 15, 2024
References Cited (367)
US 8243102B1 · Cornell · 2012 [cited by applicant]
US 8849957B1 · Boodman et al. · 2014 [cited by applicant]
US 9041739B2 · Latta et al. · 2015 [cited by applicant]
US 9088787B1 · Smith et al. · 2015 [cited by applicant]
US 9467718B1 · Newell et al. · 2016 [cited by applicant]
US 10192145B2 · Ben Himane et al. · 2019 [cited by applicant]
US 10335572B1 · Kumar · 2019 [cited by applicant]
US 10373366B2 · Forutanpour et al. · 2019 [cited by applicant]
US 10492981B1 · Kumar · 2019 [cited by applicant]
US 10504008B1 · Powers et al. · 2019 [cited by applicant]
US 10565731B1 · Reddy et al. · 2020 [cited by applicant]
US 10748302B1 · Dine et al. · 2020 [cited by applicant]
US 10852828B1 · Gatson et al. · 2020 [cited by applicant]
US 10854012B1 · Iyer et al. · 2020 [cited by applicant]
US 10957112B2 · Miranda et al. · 2021 [cited by applicant]
US 11201981B1 · Suiter et al. · 2021 [cited by applicant]
US 11227435B2 · Mohan et al. · 2022 [cited by applicant]
US 11232635B2 · Brodsky et al. · 2022 [cited by applicant]
US 11257294B2 · Zhao et al. · 2022 [cited by applicant]
US 11386627B2 · Caswell et al. · 2022 [cited by applicant]
US 11386629B2 · Miranda et al. · 2022 [cited by applicant]
US 11410395B2 · Velasquez et al. · 2022 [cited by applicant]
US 11551430B2 · Velasquez et al. · 2023 [cited by applicant]
US 11562525B2 · Joseph et al. · 2023 [cited by applicant]
US 11562542B2 · Zhang et al. · 2023 [cited by applicant]
US 11568605B2 · Shahrokni et al. · 2023 [cited by applicant]
US 11632679B2 · Shveki et al. · 2023 [cited by applicant]
US 11748963B2 · Zhang et al. · 2023 [cited by applicant]
US 11789524B2 · Brodsky et al. · 2023 [cited by applicant]
US 11790619B2 · Velasquez et al. · 2023 [cited by applicant]
US 11830149B2 · Zhao et al. · 2023 [cited by applicant]
US 11869158B2 · Caswell et al. · 2024 [cited by applicant]
US 11900547B2 · Gomez Gonzalez et al. · 2024 [cited by applicant]
US 20080303787A1 · Zheng · 2008 [cited by applicant]
US 20090031228A1 · Buchs et al. · 2009 [cited by applicant]
US 20090215536A1 · Yee et al. · 2009 [cited by applicant]
US 20090241037A1 · Hyndman · 2009 [cited by applicant]
US 20090256903A1 · Spooner et al. · 2009 [cited by applicant]
US 20100169837A1 · Hyndman · 2010 [cited by applicant]
US 20100208033A1 · Edge et al. · 2010 [cited by applicant]
US 20100257252A1 · Dougherty et al. · 2010 [cited by applicant]
US 20100287485A1 · Bertolami et al. · 2010 [cited by applicant]
US 20100321390A1 · Kim et al. · 2010 [cited by applicant]
US 20110083101A1 · Sharon et al. · 2011 [cited by applicant]
US 20110122308A1 · Duparre · 2011 [cited by applicant]
US 20110208817A1 · Toledano et al. · 2011 [cited by applicant]
US 20110254950A1 · Bibby et al. · 2011 [cited by applicant]
US 20110299736A1 · Choi et al. · 2011 [cited by applicant]
US 20120130632A1 · Bandyopadhyay et al. · 2012 [cited by applicant]
US 20120169887A1 · Zhu et al. · 2012 [cited by applicant]
US 20120188237A1 · Han et al. · 2012 [cited by applicant]
US 20120249741A1 · Maciocci et al. · 2012 [cited by applicant]
US 20120294231A1 · Finlow-Bates et al. · 2012 [cited by applicant]
US 20130002815A1 · Smoot et al. · 2013 [cited by applicant]
US 20130044128A1 · Liu et al. · 2013 [cited by applicant]
US 20130083173A1 · Geisner et al. · 2013 [cited by applicant]
US 20130141419A1 · Mount et al. · 2013 [cited by applicant]
US 20130162481A1 · Parvizi et al. · 2013 [cited by applicant]
US 20130176430A1 · Zhu et al. · 2013 [cited by applicant]
US 20130201185A1 · Kochi · 2013 [cited by applicant]
US 20130215264A1 · Soatto et al. · 2013 [cited by applicant]
US 20130222555A1 · Nagasaka et al. · 2013 [cited by applicant]
US 20130257858A1 · Na et al. · 2013 [cited by applicant]
US 20130257907A1 · Matsui · 2013 [cited by applicant]
US 20130282345A1 · McCulloch et al. · 2013 [cited by applicant]
US 20130293468A1 · Perez et al. · 2013 [cited by applicant]
US 20130321402A1 · Moore et al. · 2013 [cited by applicant]
US 20130321671A1 · Cote et al. · 2013 [cited by applicant]
US 20130321678A1 · Cote et al. · 2013 [cited by applicant]
US 20130342671A1 · Hummel et al. · 2013 [cited by applicant]
US 20140002607A1 · Shotton et al. · 2014 [cited by applicant]
US 20140003762A1 · Macnamara · 2014 [cited by applicant]
US 20140010407A1 · Sinha et al. · 2014 [cited by applicant]
US 20140097329A1 · Wadsworth · 2014 [cited by applicant]
US 20140119602A1 · Zuo · 2014 [cited by applicant]
US 20140137100A1 · Won et al. · 2014 [cited by applicant]
US 20140211855A1 · Alipour Kashi et al. · 2014 [cited by applicant]
US 20140254936A1 · Sun et al. · 2014 [cited by applicant]
US 20140254942A1 · Liu et al. · 2014 [cited by applicant]
US 20140267234A1 · Hook et al. · 2014 [cited by applicant]
US 20140282162A1 · Fein et al. · 2014 [cited by applicant]
US 20140289607A1 · Ko et al. · 2014 [cited by applicant]
US 20140306866A1 · Miller et al. · 2014 [cited by applicant]
US 20140315570A1 · Yun et al. · 2014 [cited by applicant]
US 20140368645A1 · Ahuja et al. · 2014 [cited by applicant]
US 20140372957A1 · Keane et al. · 2014 [cited by applicant]
US 20140375688A1 · Redmann et al. · 2014 [cited by applicant]
US 20150016777A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150049004A1 · Deering et al. · 2015 [cited by applicant]
US 20150049201A1 · Liu et al. · 2015 [cited by applicant]
US 20150071524A1 · Lee · 2015 [cited by applicant]
US 20150126223A1 · Lee et al. · 2015 [cited by applicant]
US 20150161476A1 · Kurz et al. · 2015 [cited by applicant]
US 20150178939A1 · Bradski et al. · 2015 [cited by applicant]
US 20150186745A1 · Martini · 2015 [cited by applicant]
US 20150187133A1 · Martini · 2015 [cited by applicant]
US 20150205126A1 · Schowengerdt · 2015 [cited by applicant]
US 20150235447A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150279081A1 · Monk et al. · 2015 [cited by applicant]
US 20150281869A1 · Ramachandran et al. · 2015 [cited by applicant]
US 20150302642A1 · Miller · 2015 [cited by applicant]
US 20150302652A1 · Miller et al. · 2015 [cited by applicant]
US 20150302656A1 · Miller et al. · 2015 [cited by applicant]
US 20150302664A1 · Miller · 2015 [cited by applicant]
US 20150302665A1 · Miller · 2015 [cited by applicant]
US 20150309264A1 · Abovitz et al. · 2015 [cited by applicant]
US 20150310664A1 · Boussard et al. · 2015 [cited by applicant]
US 20150321103A1 · Barnett et al. · 2015 [cited by applicant]
US 20160005229A1 · Lee et al. · 2016 [cited by applicant]
US 20160012643A1 · Kezele et al. · 2016 [cited by applicant]
US 20160026253A1 · Bradski et al. · 2016 [cited by applicant]
US 20160071278A1 · Leonard et al. · 2016 [cited by applicant]
US 20160086381A1 · Jung et al. · 2016 [cited by applicant]
US 20160147408A1 · Bevis et al. · 2016 [cited by applicant]
US 20160148433A1 · Petrovskaya et al. · 2016 [cited by applicant]
US 20160154821A1 · Kansal et al. · 2016 [cited by applicant]
US 20160179830A1 · Schmalstieg et al. · 2016 [cited by applicant]
US 20160180593A1 · Yang · 2016 [cited by applicant]
US 20160180602A1 · Fuchs · 2016 [cited by applicant]
US 20160196692A1 · Kjallstrom et al. · 2016 [cited by applicant]
US 20160217614A1 · Kraver et al. · 2016 [cited by applicant]
US 20160219408A1 · Yang et al. · 2016 [cited by applicant]
US 20160262253A1 · Isaacs et al. · 2016 [cited by applicant]
US 20160284314A1 · Darshan et al. · 2016 [cited by applicant]
US 20160300389A1 · Glenn, III et al. · 2016 [cited by applicant]
US 20160335275A1 · Williams et al. · 2016 [cited by applicant]
US 20160343165A1 · Park et al. · 2016 [cited by applicant]
US 20160358383A1 · Gauglitz et al. · 2016 [cited by applicant]
US 20160360111A1 · Thivent et al. · 2016 [cited by applicant]
US 20160370971A1 · Hackett et al. · 2016 [cited by applicant]
US 20160381118A1 · Andrews et al. · 2016 [cited by applicant]
US 20170031160A1 · Popovich et al. · 2017 [cited by applicant]
US 20170061696A1 · Li et al. · 2017 [cited by applicant]
US 20170076408A1 · D'Souza et al. · 2017 [cited by applicant]
US 20170091996A1 · Wei et al. · 2017 [cited by applicant]
US 20170094227A1 · Williams et al. · 2017 [cited by applicant]
US 20170134909A1 · Gu et al. · 2017 [cited by applicant]
US 20170185823A1 · Gold et al. · 2017 [cited by applicant]
US 20170192515A1 · Menadeva et al. · 2017 [cited by applicant]
US 20170195564A1 · Appia et al. · 2017 [cited by applicant]
US 20170208109A1 · Akselrod et al. · 2017 [cited by applicant]
US 20170236037A1 · Rhoads et al. · 2017 [cited by applicant]
US 20170237789A1 · Harner et al. · 2017 [cited by applicant]
US 20170243352A1 · Kutliroff et al. · 2017 [cited by applicant]
US 20170270713A1 · Dooley et al. · 2017 [cited by applicant]
US 20170336511A1 · Nerurkar et al. · 2017 [cited by applicant]
US 20170345167A1 · Ard et al. · 2017 [cited by applicant]
US 20170345215A1 · Khedkar et al. · 2017 [cited by applicant]
US 20170352192A1 · Petrovskaya et al. · 2017 [cited by applicant]
US 20170358140A1 · Kohler et al. · 2017 [cited by applicant]
US 20170371024A1 · Ivanov et al. · 2017 [cited by applicant]
US 20180000547A1 · Kang et al. · 2018 [cited by applicant]
US 20180012074A1 · Holz et al. · 2018 [cited by applicant]
US 20180045963A1 · Hoover et al. · 2018 [cited by applicant]
US 20180053284A1 · Rodriguez et al. · 2018 [cited by applicant]
US 20180053315A1 · Ard et al. · 2018 [cited by applicant]
US 20180082156A1 · Jin et al. · 2018 [cited by applicant]
US 20180089834A1 · Spizhevoy et al. · 2018 [cited by applicant]
US 20180114065A1 · Zeng · 2018 [cited by applicant]
US 20180122143A1 · Ellwood, Jr. · 2018 [cited by applicant]
US 20180164877A1 · Miller et al. · 2018 [cited by applicant]
US 20180189556A1 · Shamir et al. · 2018 [cited by applicant]
US 20180213359A1 · Reinhardt et al. · 2018 [cited by applicant]
US 20180218222A1 · Alrabeiah et al. · 2018 [cited by applicant]
US 20180245927A1 · Frish et al. · 2018 [cited by applicant]
US 20180261012A1 · Mullins et al. · 2018 [cited by applicant]
US 20180268237A1 · Stanimirovic et al. · 2018 [cited by applicant]
US 20180268582A1 · Schneider et al. · 2018 [cited by applicant]
US 20180268611A1 · Nourai et al. · 2018 [cited by applicant]
US 20180284802A1 · Tsai et al. · 2018 [cited by applicant]
US 20180285052A1 · Eade et al. · 2018 [cited by applicant]
US 20180286116A1 · Babu · 2018 [cited by applicant]
US 20180293771A1 · Piemonte et al. · 2018 [cited by applicant]
US 20180304153A1 · Hohjoh et al. · 2018 [cited by applicant]
US 20180307303A1 · Powderly et al. · 2018 [cited by applicant]
US 20180308377A1 · Pena-Rios et al. · 2018 [cited by applicant]
US 20180315248A1 · Bastov et al. · 2018 [cited by applicant]
US 20190005725A1 · Oonishi · 2019 [cited by applicant]
US 20190027267A1 · Hayashi et al. · 2019 [cited by applicant]
US 20190035047A1 · Lim et al. · 2019 [cited by applicant]
US 20190065814A1 · Morein et al. · 2019 [cited by applicant]
US 20190080467A1 · Hirzer et al. · 2019 [cited by applicant]
US 20190114798A1 · Afrouzi et al. · 2019 [cited by applicant]
US 20190114802A1 · Lazarow · 2019 [cited by applicant]
US 20190139311A1 · Petrovskaya et al. · 2019 [cited by applicant]
US 20190147341A1 · Rabinovich et al. · 2019 [cited by applicant]
US 20190188474A1 · Zahnert et al. · 2019 [cited by applicant]
US 20190197785A1 · Tate-Gans et al. · 2019 [cited by applicant]
US 20190199882A1 · Han · 2019 [cited by applicant]
US 20190206258A1 · Chang et al. · 2019 [cited by applicant]
US 20190236797A1 · Thyagharajan et al. · 2019 [cited by applicant]
US 20190287311A1 · Bhatnagar et al. · 2019 [cited by applicant]
US 20190301873A1 · Prasser et al. · 2019 [cited by applicant]
US 20190310761A1 · Agarawala et al. · 2019 [cited by applicant]
US 20190313059A1 · Agarawala et al. · 2019 [cited by applicant]
US 20190340831A1 · Scarfone et al. · 2019 [cited by applicant]
US 20190355169A1 · Sapienza et al. · 2019 [cited by applicant]
US 20190362546A1 · Wayenberg · 2019 [cited by applicant]
US 20190384379A1 · Huh · 2019 [cited by applicant]
US 20190385370A1 · Boyapalle et al. · 2019 [cited by applicant]
US 20190388182A1 · Kumar et al. · 2019 [cited by applicant]
US 20200005486A1 · Sinha et al. · 2020 [cited by applicant]
US 20200033463A1 · Lee et al. · 2020 [cited by applicant]
US 20200034624A1 · Sharma et al. · 2020 [cited by applicant]
US 20200051328A1 · Mohan et al. · 2020 [cited by applicant]
US 20200066050A1 · Ha et al. · 2020 [cited by applicant]
US 20200074739A1 · Stauber et al. · 2020 [cited by applicant]
US 20200090407A1 · Miranda et al. · 2020 [cited by applicant]
US 20200097770A1 · Sommer et al. · 2020 [cited by applicant]
US 20200111255A1 · Brodsky et al. · 2020 [cited by applicant]
US 20200126252A1 · Iyer et al. · 2020 [cited by applicant]
US 20200126256A1 · Sinha et al. · 2020 [cited by applicant]
US 20200126309A1 · Moroze et al. · 2020 [cited by applicant]
US 20200134366A1 · Xu et al. · 2020 [cited by applicant]
US 20200175766A1 · Gawrys et al. · 2020 [cited by applicant]
US 20200177870A1 · Tadi et al. · 2020 [cited by applicant]
US 20200211286A1 · Kelsey et al. · 2020 [cited by applicant]
US 20200211290A1 · Choi et al. · 2020 [cited by applicant]
US 20200252233A1 · O'Keeffe · 2020 [cited by applicant]
US 20200271450A1 · Gorur Sheshagiri et al. · 2020 [cited by applicant]
US 20200342670A1 · Nattinger et al. · 2020 [cited by applicant]
US 20200349730A1 · Chen · 2020 [cited by applicant]
US 20200364901A1 · Choudhuri et al. · 2020 [cited by applicant]
US 20200364937A1 · Selbrede · 2020 [cited by applicant]
US 20200372672A1 · Schonberger et al. · 2020 [cited by applicant]
US 20200380263A1 · Yang et al. · 2020 [cited by applicant]
US 20200380769A1 · Liu et al. · 2020 [cited by applicant]
US 20200394012A1 · Wright, Jr. et al. · 2020 [cited by applicant]
US 20200401617A1 · Spiegel et al. · 2020 [cited by applicant]
US 20210019909A1 · Wang et al. · 2021 [cited by applicant]
US 20210049360A1 · Yildiz et al. · 2021 [cited by applicant]
US 20210065455A1 · Beith et al. · 2021 [cited by applicant]
US 20210074072A1 · Desai et al. · 2021 [cited by applicant]
US 20210103449A1 · Terpstra et al. · 2021 [cited by applicant]
US 20210105340A1 · Grozdanov et al. · 2021 [cited by applicant]
US 20210110614A1 · Shahrokni et al. · 2021 [cited by applicant]
US 20210110615A1 · Zhao et al. · 2021 [cited by applicant]
US 20210112427A1 · Shveki et al. · 2021 [cited by applicant]
US 20210125414A1 · Berkebile · 2021 [cited by applicant]
US 20210134064A1 · Shahrokni et al. · 2021 [cited by applicant]
US 20210142580A1 · Caswell et al. · 2021 [cited by applicant]
US 20210174596A1 · Zhang et al. · 2021 [cited by applicant]
US 20210209859A1 · Miranda et al. · 2021 [cited by applicant]
US 20210256755A1 · Joseph et al. · 2021 [cited by applicant]
US 20210256766A1 · Muhlethaler et al. · 2021 [cited by applicant]
US 20210256767A1 · Velasquez et al. · 2021 [cited by applicant]
US 20210256768A1 · Zhao et al. · 2021 [cited by applicant]
US 20210264620A1 · Ramasamy et al. · 2021 [cited by applicant]
US 20210264685A1 · Velasquez et al. · 2021 [cited by applicant]
US 20210295266A1 · McKee et al. · 2021 [cited by applicant]
US 20210315464A1 · Sol et al. · 2021 [cited by applicant]
US 20210343087A1 · Gomez Gonzalez et al. · 2021 [cited by applicant]
US 20210358150A1 · Lin et al. · 2021 [cited by applicant]
US 20220036648A1 · Wang · 2022 [cited by applicant]
US 20220092852A1 · Mohan et al. · 2022 [cited by applicant]
US 20220101607A1 · Brodsky et al. · 2022 [cited by applicant]
US 20220130120A1 · Zhao et al. · 2022 [cited by applicant]
US 20220292789A1 · Caswell et al. · 2022 [cited by applicant]
US 20220358733A1 · Velasquez et al. · 2022 [cited by applicant]
US 20230119217A1 · Velasquez et al. · 2023 [cited by applicant]
US 20230119305A1 · Zhang et al. · 2023 [cited by applicant]
US 20230127303A1 · Shahrokni et al. · 2023 [cited by applicant]
US 20230209373A1 · Shveki et al. · 2023 [cited by applicant]
US 20230222731A1 · Joseph et al. · 2023 [cited by applicant]
US 20240029369A1 · Zhang et al. · 2024 [cited by applicant]
US 20240062491A1 · Velasquez et al. · 2024 [cited by applicant]
US 20240069628A1 · Brodsky et al. · 2024 [cited by applicant]
US 20240087258A1 · Zhao et al. · 2024 [cited by applicant]
CA 2788836A1 · 2011 [cited by applicant]
CN 102216957A · 2011 [cited by applicant]
CN 103460255A · 2013 [cited by applicant]
CN 104508600A · 2015 [cited by applicant]
CN 106062862A · 2016 [cited by applicant]
CN 106663411A · 2017 [cited by applicant]
CN 106937531A · 2017 [cited by applicant]
CN 107810463A · 2018 [cited by applicant]
CN 107924584A · 2018 [cited by applicant]
CN 108398921A · 2018 [cited by applicant]
CN 108885522A · 2018 [cited by applicant]
CN 109765992A · 2019 [cited by applicant]
CN 112074876A · 2020 [cited by applicant]
CN 112189335A · 2021 [cited by applicant]
EP 2359333A1 · 2011 [cited by applicant]
EP 2808842B1 · 2017 [cited by applicant]
EP 4046139A1 · 2022 [cited by applicant]
JP 2013141049A · 2013 [cited by applicant]
JP 2015079490A · 2015 [cited by applicant]
JP 2016522463A · 2016 [cited by applicant]
JP 2017107604A · 2017 [cited by applicant]
JP 2017529635A · 2017 [cited by applicant]
WO WO2012126500A1 · 2012 [cited by applicant]
WO WO2014164901A1 · 2014 [cited by applicant]
WO WO2015161307A1 · 2015 [cited by applicant]
WO WO2015192117A1 · 2015 [cited by applicant]
WO WO2016077798A1 · 2016 [cited by applicant]
WO WO2017136833A1 · 2017 [cited by applicant]
WO WO2018125428A1 · 2018 [cited by applicant]
WO WO2019046774A1 · 2019 [cited by applicant]
WO WO2019210284A1 · 2019 [cited by applicant]
WO WO2019221580A1 · 2019 [cited by applicant]
WO WO2019221800A1 · 2019 [cited by applicant]
Extended European Search Report dated Sep. 28, 2023 in connection with European Application No. 20886597.2. [cited by applicant]
International Search Report and Written Opinion for International Application No. PCT/US2020/059975, mailed Feb. 4, 2021. [cited by applicant]
[No Author Listed], Axis-angle representation—Wikipedia. 6 pages. Last edited on Feb. 2, 2022. URL:https://en.wikipedia.org/wiki/Axis%E2%80%93angle_representation [retrieved on Feb. 28, 2022]. [cited by applicant]
[No Author Listed], Code release for “learning to find good correspondences” CVPR 2018. GitHub. Sep. 30, 2020. 4 pages. URL:https://github.com/vgc-uvic/learned-correspondence-release [retrieved on Feb. 22, 2022]. [cited by applicant]
[No Author Listed], Combain Location API—API Reference. 14 pages. URL:https://combain.com/api/#combain-location-api [retrieved on Feb. 24, 2021]. [cited by applicant]
[No Author Listed], Content Persistence Fundamentals. Magic Leap, Inc. Oct. 23, 2019. URL:https://developer.magicleap.com/en-us/learn/guides/content-persistence-fundamentals [retrieved on Dec. 31, 2020]. 5 pages. [cited by applicant]
[No Author Listed], Course (navigation)—Wikipedia. 3 pages. Last edited on Nov. 4, 2021. URL:https://en.wikipedia.org/wiki/Course_(navigation) [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], Geohash a Ing/at coordinate using hibert space filling curves. GitHub. Apr. 1, 2020. 9 pages. URL:https://github.com/tammoippen/geohash-hilbert [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], GitHub—gmplot/gmplot: Plot data on Google Maps, the easy way. Oct. 14, 2021. 2 pages. URL:https://github.com/vgm64/gmplot [retrieved on Feb. 28, 2022]. [cited by applicant]
[No Author Listed], Haversine formula—Wikipedia. 5 pages. Last edited on Jan. 11, 2022. URL:https://en.wikipedia.org/wiki/Haversine_formula [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], Kismet (software). Wikipedia. Last edited on Jan. 27, 2021. 3 pages. URL:https://en.wikipedia.org/wiki/Kismet_(software) [retrieved on Feb. 24, 2021]. [cited by applicant]
[No Author Listed], Points of the Compass—Wikipedia. Last edited on Dec. 4, 2021. 16 pages. URL:https://en.wikipedia.org/wiki/Points_of_the_compass [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], Progressive Web Apps. Google Developers. 2022, 5 pages. URL:https://web.dev/progressive-web-apps [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], S2 Geometry. 3 pages. URL:http://s2geometry.io/ [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], Skid (aerodynamics)—Wikipedia. 2 pages. Last edited on Jun. 17, 2020. URL:https://en.wikipedia.org/wiki/Skid_(aerodynamic) [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], sklearn.neighbors.BallTree. 2022. 4 pages. URL:https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.BallTree.html [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], Slip (aerodynamics)—Wikipedia. 5 pages. Last edited on Aug. 22, 2021. URL:https://en.wikipedia.org/wiki/Slip_(aerodynamic) [retrieved on Feb. 24, 2022]. [cited by applicant]
[No Author Listed], The difference between virtual reality, Augmented Reality and Mixed Reality. Forbes. Feb. 2, 2018. 5 pages. URL:https://www.forbes.com/sites/quora/2018/02/02/the-difference-between-virtual-reality-au… [cited by applicant]
[No Author Listed], Wi-Fi Location-Based Services 4.1 Design Guide. Jan. 30, 2014. 11 pages. URL:https://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/Mobility/WiFiLBS-DG/wifich2.html. [cited by applicant]
[No Author Listed], Wigle: Wireless Network Mapping. 2021. 2 pages. URL:https://wigle.net [retrieved on Feb. 24, 2021]. [cited by applicant]
[No Author Listed], Wind Triangle—Wikipedia. 2 pages. Last edited on Nov. 16, 2021. URL:https://en.wikipedia.org/wiki/Wind_triangle [retrieved on Feb. 24, 2022]. [cited by applicant]
Balntas et al., HPatches: A benchmark and evaluation of handcrafted and learned local descriptors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2017. pp. 5173-5182. [cited by applicant]
Bansal et al., Blur image detection using Laplacian operator and Open-CV. 2016 International Conference System Modeling & Advancement in Research Trends (SMART). Nov. 2016, pp. 63-67. [cited by applicant]
Bleser et al., Cognitive learning, monitoring and assistance of industrial workflows using egocentric sensor networks. PloS one. Jun. 30, 2015;10(6):e0127769. 41 pages. [cited by applicant]
Brachmann et al., Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses. arXiv:1905.04132v2. Jul. 31, 2019. 17 pages. [cited by applicant]
Brief, Mobile Image Blur Detection with Machine Learning. May 17, 2019. 14 pages. URL:https://medium.com/snapaddy-tech-blog/mobile-image-blur-detection-with-machine-learning-c0b703eab7de. [cited by applicant]
Brodsky et al., Rendering Location Specific Virtual Content In Any Location, U.S. Appl. No. 18/460,873, filed Sep. 5, 2023. [cited by applicant]
Cadena et al., Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age. IEEE Transactions on Robotics. Dec. 2016;32(6):1309-1332. [cited by applicant]
Chatterjee, How to Leverage Geolocation Capabilities to Deliver a Top Notch Mobile App. Jul. 19, 2017. 5 pages. URL:https://www.fingent.com/blog/how-to-leverage-geo-location-capabilities-to-deliver-a-top-notch-mobile-ap… [cited by applicant]
Dang et al., Eigendecomposition-free training of deep networks with zero eigenvalue-based losses. arXiv: 1803.08071. Mar. 26, 2018. 25 pages. [cited by applicant]
Gidaris et al, Unsupervised representation learning by predicting image rotations. arXiv:1803.07728. Mar. 21, 2018. 16 pages. [cited by applicant]
Henniges, Current approaches of Wifi Positioning. Service-Centric Networking Seminar. WS2011/2012. 8 pages. [cited by applicant]
Henry et al., RGB-D mapping: Using Kinect-style depth cameras for dense 30 modeling of indoor environments. The International Journal of Robotics Research. Feb. 10, 2012. 26 pages. URL:http://citeseerx.ist.psu.edu/viewd… [cited by applicant]
Huang et al., Efficient, Generalized Indoor WiFi GraphSLAM. IEEE International Conference on Robotics and Automation. 2011. 3 pages. doi: 10.1109/ICRA.2011.5979643. [cited by applicant]
Ito et al., W-RGB-D: Floor-Plan-Based Indoor Global Localization Using a Depth Camera and WiFi. 2014 IEEE international conference on robotics and automation (ICRA). May 2014, pp. 417-422. [cited by applicant]
Kurz et al., Representative feature descriptor sets for robust handheld camera localization. 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Nov. 5, 2012. pp. 65-70. [cited by applicant]
Larsson et al., Fine-grained segmentation networks: Self-supervised segmentation for improved long-term visual localization. arXiv: 1908.06387v1. Aug. 18, 2019. 13 pages. [cited by applicant]
Lynen et al., Get Out of My Lab: Large-scale, Real-Time Visual-Inertial Localization. Robotics: Science and Systems. Jul. 2015. 10 pages. [cited by applicant]
Lynen et al., Large-scale, real-time visual-inertial localization revisited. arXiv preprint arXiv: 1907.00338v1. Jun. 30, 2019. 21 pages. [cited by applicant]
Mirowski et al., Depth camera SLAM on a low-cost WiFi mapping robot. Apr. 2012. 7 pages. doi:10.1109/TePRA.2012.6215673. [cited by applicant]
Mohanna et al., Optimization of Music algorithm for angle of arrival estimation in wireless communications. NRIAG Journal of Astronomy and Geophysics. 2013:2:116-124. [cited by applicant]
Mueller, Fast In-memory spatial radius queries with Python. Aug. 9, 2017. 8 pages. URL:https://medium.com/@alexander.mueller/experiments-with-in-memory-spatial-radius-queries-in-python-e40c9e66cf63 [retrieved on Feb. 24… [cited by applicant]
Panzarino, What exactly WiFiSLAM is, and why Apple acquired it. Mar. 25, 2013. URL:https://thenextweb.com/apple/2013/03/26/what-exactly-wifislam-is-and-why-apple-acquired-it [retrieved Feb. 24, 2021]. [cited by applicant]
Pertuz et al., Analysis of focus measure operators for shape-from-focus. Pattern Recognition. May 2013;46:1415-32. [cited by applicant]
Qi et al., Pointnet: Deep learning on point sets for 3d classification and segmentation. arXiv:1612.00593. Apr. 10, 2017. 19 pages. [cited by applicant]
Rabinovich et al., Lumin OS & Lumin SDK: past, present and future. Magic Leap, Inc. Apr. 2, 2019. URL:https://www.magicleap.com/en-us/news/product-updates/lumin-os-and-lumin-sdk-update [retrieved on Dec. 31, 2020]. 9 pa… [cited by applicant]
Raskar et al., Interacting with spatially augmented reality. Proceedings of the 1st international conference on Computer graphics, virtual reality and visualisation Nov. 2001, pp. 101-108. [cited by applicant]
Snavely et al., Skeletal graphs for efficient structure from motion. IEEE Conference on Computer Vision and Pattern Recognition. Jun. 23, 2008. 11 pages. URL:http://www.cs.cornell.edu/˜snavely/projects/skeletalset. [cited by applicant]
Stobing, How to Add Website Links to the Windows 10 Start Menu. Howtogeek.com. 2016. 9 pages. URL:https://www.howtogeek.com/237951/how-to-add-website-links-to-the-windows-10-start-menu [Last accessed Jul. 11, 2022]. [cited by applicant]
Sturari et al., Robust and affordable retail customer profiling by vision and radio beacon sensor fusion. Pattern Recognition Letters. Oct. 1, 2016;81:30-40. [cited by applicant]
Sweeney et al., Efficient computation of absolute pose for gravity-aware augmented reality. 2015 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Sep. 29, 2015, pp. 19-24. [cited by applicant]
Taira et al., InLoc: Indoor visual localization with dense matching and view synthesis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018. pp. 7199-7209. [cited by applicant]
Tang, Applying Deep Learning to Detect Blurry Images. Dec. 12, 2017. 6 pages. URL:https://tangming2008.github.io/neural network/tensor flow/classification/Applying-Deep-Learning-to-Detect-Blurry-Images/. [cited by applicant]
Vogt, Real-Time Augmented Reality for Image-Guided Interventions. Doctoral dissertation, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). Feb. 18, 2009. 48 pages. URL:https://opus4.kobv.de/opus4-fau/files/1235/s… [cited by applicant]
Wikidan61, Normalization (image processing). Wikipedia. Apr. 23, 2018. 2 pages. URL:https://en.wikipedia.org/w/index.php?title=Normalization_(image_processing)&oldid=837856854 [retrieved on Jun. 16, 2023]. [cited by applicant]
Willaredt, WiFi and Cell-ID based positioning—Protocols, Standards and Solutions. SNET Project WT. Jan. 26, 2011. 10 pages. [cited by applicant]
Xiong et al., A Diversified Generative Latent Variable Model for WiFi-SLAM. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17). Feb. 2017;31:3841-3847. [cited by applicant]
Yi et al., Learning to find good correspondences. arXiv: 1711.05971. May 21, 2018. 13 pages. [cited by applicant]
Velasquez et al., Cross Reality System With Accurate Shared Maps, U.S. Appl. No. 18/457,314, filed Aug. 28, 2023. [cited by applicant]
Zhang et al., Cross Reality System With Simplified Programming Of Virtual Content, U.S. Appl. No. 18/353,775, filed Jul. 17, 2023. [cited by applicant]
Zhao et al., Cross Reality System With Prioritization Of Geolocation Information For Localization, U.S. Appl. No. 18/510,623, filed Nov. 15, 2023. [cited by applicant]
Chinese Office Action dated Mar. 27, 2024 in connection with Chinese Application No. 202080078518.0. [cited by applicant]
Gomez Gonzalez et al., Cross Reality System For Large Scale Environments, U.S. Appl. No. 18/396,682, filed Dec. 26, 2023. [cited by applicant]
Joseph et al., Cross Reality System With Map Processing Using Multi-Resolution Frame Descriptors, U.S. Appl. No. 18/609,101, filed Mar. 19, 2024. [cited by applicant]
Mohan et al., Cross Reality System, U.S. Appl. No. 18/612,614 filed Mar. 21, 2024. [cited by applicant]
Sarlin et al., From coarse to fine: Robust hierarchical localization at large scale. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019. pp 12708-12717. doi: 10.1109/cvpr.2019.01300. [cited by applicant]
Cited By (2)
US 12,499,618 US 12,694,586