IP Library Granted Patent US 12,271,517
Granted Patent B1
US 12,271,517 · App. 18/478,352 · Granted Apr 8, 2025

Bending-assisted calibration for extended reality tracking

Inventors: Thomas Faeulhammer (Vienna, AT); Matthias Kalkgruber (Vienna, AT); Thomas Muttenthaler (Vienna, AT); Tiago Miguel Pereira Torres (Vienna, AT); Daniel Wolf (Mödling, AT)
Assignee: Snap Inc.
G06F3/012A61B5/16G06T17/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,271,517
App. No.
18/478,352
Filed
Sep 29, 2023
Granted
Apr 8, 2025
Kind
B1
Examiner
LIN, HANG
Art Unit
2626
USPC
345/156
Abstract

Bending data is used to facilitate tracking operations of an extended reality (XR) device, such as hand tracking or other object tracking operations. The XR device obtains bending data indicative of bending of the XR device to accommodate a body part of a user wearing the XR device. The XR device determines, based on the bending data, whether to use previously identified biometric data in a tracking operation. A mode of the XR device is selected responsive to determining whether to use the previously identified biometric data. The selected mode is used to initialize the tracking operation. The selected mode may be a first mode in which the previously identified biometric data is used in the tracking operation or a second mode in which the previously identified biometric data is not used in the tracking operation.

Claims (41)

1. A method comprising:

obtaining bending data indicative of bending of an extended reality (XR) device to accommodate a body part of a user wearing the XR device;

determining, based on the bending data, whether to use previously identified biometric data in a tracking operation;

responsive to determining whether to use the previously identified biometric data, selecting a mode of the XR device, the selected mode being a first mode or a second mode, the first mode being based on determining to use the previously identified biometric data in the tracking operation and the second mode being based on determining not to use the previously identified biometric data in the tracking operation; and

using the selected mode to initialize the tracking operation.

2. The method of claim 1 , wherein the tracking operation comprises tracking a feature of the user wearing the XR device.

3. The method of claim 2 , wherein the feature is a hand of the user wearing the XR device, and the previously identified biometric data comprises a previously obtained hand feature calibration.

4. The method of claim 1 , wherein the previously identified biometric data comprises a hand feature estimate generated during a previous user session.

5. The method of claim 1 , wherein the bending data comprises at least one of: a measurement of the bending of the XR device generated by at least one sensor of the XR device; or an estimate of the bending of the XR device.

6. The method of claim 1 , wherein the bending data is first bending data, and wherein the determining whether to use the previously identified biometric data in the tracking operation comprises:

comparing the first bending data to second bending data associated with the previously identified biometric data.

7. The method of claim 6 , wherein the second bending data comprises a measurement or estimate of the bending of the XR device generated during a previous user session.

8. The method of claim 6 , wherein the selected mode is the first mode in which the previously identified biometric data is selected for use in the initializing of the tracking operation, and the first mode is selected based on determining that the first bending data matches the second bending data.

9. The method of claim 8 , further comprising:

identifying, based on the determining that the first bending data matches the second bending data, that no calibration operation is required for the user.

10. The method of claim 9 , further comprising:

adjusting the previously identified biometric data during the tracking operation.

11. The method of claim 10 , wherein the previously identified biometric data is adjusted so as to refine the biometric data across multiple user sessions.

12. The method of claim 8 , wherein the determining that the first bending data matches the second bending data comprises determining that the first bending data is within an acceptable bending range of the second bending data.

13. The method of claim 8 , further comprising:

detecting, based on the determining that the first bending data matches the second bending data, that the user corresponds to a previous user of the XR device, the previous user being associated with the second bending data and the previously identified biometric data.

14. The method of claim 6 , wherein the selected mode is the second mode in which a calibration operation is performed for the user, and the second mode is selected based on determining that the first bending data does not match the second bending data.

15. The method of claim 14 , further comprising:

performing the calibration operation to obtain new biometric data for use in the tracking operation; and

associating the new biometric data with the first bending data.

16. The method of claim 14 , further comprising:

detecting, based on the determining that the first bending data does not match the second bending data, that the user does not correspond to a previous user of the XR device, the previous user being associated with the second bending data and the previously identified biometric data.

17. A system comprising:

at least one processor; and

at least one memory component storing instructions that, when executed by the at least one processor, configure the system to perform operations comprising:

obtaining bending data indicative of bending of an extended reality (XR) device to accommodate a body part of a user wearing the XR device;

determining, based on the bending data, whether to use previously identified biometric data in a tracking operation;

responsive to determining whether to use the previously identified biometric data, selecting a mode of the XR device, the selected mode being a first mode or a second mode, the first mode being based on determining to use the previously identified biometric data in the tracking operation and the second mode being based on determining not to use the previously identified biometric data in the tracking operation; and

using the selected mode to initialize the tracking operation.

18. The system of claim 17 , wherein the tracking operation comprises tracking a feature of the user wearing the XR device.

19. The system of claim 18 , wherein the feature is a hand of the user wearing the XR device, and the previously identified biometric data comprises a previously obtained hand feature calibration.

20. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by at least one processor, cause the at least one processor to perform operations comprising:

obtaining bending data indicative of bending of an extended reality (XR) device to accommodate a body part of a user wearing the XR device;

determining, based on the bending data, whether to use previously identified biometric data in a tracking operation;

responsive to determining whether to use the previously identified biometric data, selecting a mode of the XR device, the selected mode being a first mode or a second mode, the first mode being based on determining to use the previously identified biometric data in the tracking operation and the second mode being based on determining not to use the previously identified biometric data in the tracking operation; and

using the selected mode to initialize the tracking operation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 30, 2024
From: FAEULHAMMER, THOMAS; KALKGRUBER, MATTHIAS; MUTTENTHALER, THOMAS; PEREIRA TORRES, TIAGO MIGUEL; WOLF, DANIEL
To: SNAP INC.
Reel/Frame 069697/0538 →
References Cited (108)
US 7971156B2 · Albertson et al. · 2011 [cited by applicant]
US 7996793B2 · Latta et al. · 2011 [cited by applicant]
US 8487938B2 · Latta et al. · 2013 [cited by applicant]
US 8856691B2 · Geisner et al. · 2014 [cited by applicant]
US 9225897B1 · Sehn et al. · 2015 [cited by applicant]
US 9230160B1 · Kanter · 2016 [cited by applicant]
US 9276886B1 · Samaranayake · 2016 [cited by applicant]
US 9705831B2 · Spiegel · 2017 [cited by applicant]
US 9742713B2 · Spiegel et al. · 2017 [cited by applicant]
US 9876788B1 · Ziraknejad et al. · 2018 [cited by applicant]
US 10102423B2 · Shaburov et al. · 2018 [cited by applicant]
US 10284508B1 · Allen et al. · 2019 [cited by applicant]
US 10439972B1 · Spiegel et al. · 2019 [cited by applicant]
US 10509466B1 · Miller et al. · 2019 [cited by applicant]
US 10514876B2 · Sehn · 2019 [cited by applicant]
US 10579869B1 · Xiong et al. · 2020 [cited by applicant]
US 10591730B2 · Rodriguez, II et al. · 2020 [cited by applicant]
US 10614855B2 · Huang · 2020 [cited by applicant]
US 10748347B1 · Li et al. · 2020 [cited by applicant]
US 10958608B1 · Allen et al. · 2021 [cited by applicant]
US 10962809B1 · Castañeda · 2021 [cited by applicant]
US 10996846B2 · Robertson et al. · 2021 [cited by applicant]
US 10997787B2 · Ge et al. · 2021 [cited by applicant]
US 11012390B1 · Al Majid et al. · 2021 [cited by applicant]
US 11030454B1 · Xiong et al. · 2021 [cited by applicant]
US 11036368B1 · Al Majid et al. · 2021 [cited by applicant]
US 11062498B1 · Voss et al. · 2021 [cited by applicant]
US 11087728B1 · Canberk et al. · 2021 [cited by applicant]
US 11092998B1 · Castañeda et al. · 2021 [cited by applicant]
US 11106342B1 · Al Majid et al. · 2021 [cited by applicant]
US 11126206B2 · Meisenholder et al. · 2021 [cited by applicant]
US 11143867B2 · Rodriguez, II · 2021 [cited by applicant]
US 11169600B1 · Canberk et al. · 2021 [cited by applicant]
US 11227626B1 · Krishnan Gorumkonda et al. · 2022 [cited by applicant]
US 11294188B1 · Birklbauer et al. · 2022 [cited by applicant]
US 11307747B2 · Dancie et al. · 2022 [cited by applicant]
US 11531402B1 · Stolzenberg · 2022 [cited by applicant]
US 11546505B2 · Canberk · 2023 [cited by applicant]
US 20090012788A1 · Gilbert et al. · 2009 [cited by applicant]
US 20110301934A1 · Tardif · 2011 [cited by applicant]
US 20140018166A1 · Guild et al. · 2014 [cited by applicant]
US 20140171036A1 · Simmons · 2014 [cited by applicant]
US 20150120293A1 · Wohlert et al. · 2015 [cited by applicant]
US 20150370320A1 · Connor · 2015 [cited by applicant]
US 20170123487A1 · Hazra et al. · 2017 [cited by applicant]
US 20170277684A1 · Dharmarajan Mary · 2017 [cited by applicant]
US 20170277685A1 · Takumi · 2017 [cited by applicant]
US 20170351910A1 · Elwazer et al. · 2017 [cited by applicant]
US 20180158370A1 · Pryor · 2018 [cited by applicant]
US 20190146219A1 · Rodriguez, II · 2019 [cited by applicant]
US 20190183430A1 · Alphonse et al. · 2019 [cited by applicant]
US 20200034739A1 · Chung et al. · 2020 [cited by applicant]
US 20200043236A1 · Miller et al. · 2020 [cited by applicant]
US 20210011612A1 · Dancie et al. · 2021 [cited by applicant]
US 20210074016A1 · Li et al. · 2021 [cited by applicant]
US 20210166732A1 · Shaburova et al. · 2021 [cited by applicant]
US 20210174034A1 · Retek et al. · 2021 [cited by applicant]
US 20210241529A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210303075A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210303077A1 · Anvaripour et al. · 2021 [cited by applicant]
US 20210303140A1 · Mourkogiannis · 2021 [cited by applicant]
US 20210382564A1 · Blachly et al. · 2021 [cited by applicant]
US 20210397000A1 · Rodriguez, II · 2021 [cited by applicant]
US 20210405761A1 · Canberk · 2021 [cited by applicant]
US 20220188539A1 · Chan et al. · 2022 [cited by applicant]
US 20220206588A1 · Canberk et al. · 2022 [cited by applicant]
US 20220300730A1 · Eirinberg et al. · 2022 [cited by applicant]
US 20220300731A1 · Eirinberg et al. · 2022 [cited by applicant]
US 20220301231A1 · Eirinberg et al. · 2022 [cited by applicant]
US 20220326781A1 · Hwang et al. · 2022 [cited by applicant]
US 20220334649A1 · Hwang · 2022 [cited by examiner]
US 20220374505A1 · Katz · 2022 [cited by examiner]
US 20220375174A1 · Arya et al. · 2022 [cited by applicant]
US 20220385881A1 · Wu et al. · 2022 [cited by applicant]
CN 103049761 · 2016 [cited by applicant]
CN 117425889 · 2024 [cited by applicant]
EP 3707693 · 2020 [cited by applicant]
EP 3834109 · 2021 [cited by applicant]
EP 4172726 · 2023 [cited by applicant]
EP 4172730 · 2023 [cited by applicant]
KR 20220158824 · 2022 [cited by applicant]
WO 2016168591 · 2016 [cited by applicant]
WO 2019018396 · 2019 [cited by applicant]
WO 2019094618 · 2019 [cited by applicant]
WO 2020161456 · 2020 [cited by applicant]
WO 2022005687 · 2022 [cited by applicant]
WO 2022005693 · 2022 [cited by applicant]
WO 2022060549 · 2022 [cited by applicant]
WO 2022066578 · 2022 [cited by applicant]
WO 2022132381 · 2022 [cited by applicant]
WO 2022146678 · 2022 [cited by applicant]
WO 2022198182 · 2022 [cited by applicant]
WO 2022216784 · 2022 [cited by applicant]
WO 2022225761 · 2022 [cited by applicant]
WO 2022245765 · 2022 [cited by applicant]
WO 2022246382 · 2022 [cited by applicant]
WO 2023031633A1 · 2023 [cited by applicant]
“International Application Serial No. PCT/US2022/072338, International Search Report mailed Aug. 23, 2022”, 5 pgs. [cited by applicant]
“International Application Serial No. PCT/US2022/072338, Written Opinion mailed Aug. 23, 2022”, 6 pgs. [cited by applicant]
Ferro, M, “A Sensing Seat for Human Authentication”, IEEE Transactions on Information Forensics and Security, IEEE, USA, vol. 4, No. 3, (Sep. 1, 2009), 451-459. [cited by applicant]
“International Application Serial No. PCT US2022 072338, International Preliminary Report on Patentability mailed Nov. 30, 2023”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/489,394, Non Final Office Action mailed Dec. 29, 2023”, 42 pgs. [cited by applicant]
“U.S. Appl. No. 17/489,394, Response filed Mar. 19, 2024 to Non Final Office Action mailed Dec. 29, 2023”, 11 pgs. [cited by applicant]
“U.S. Appl. No. 17/489,394, Final Office Action mailed Mar. 29, 2024”, 50 pgs. [cited by applicant]
“U.S. Appl. No. 17/489,394, Response filed Jun. 28, 2024 to Final Office Action mailed Mar. 29, 2024”, 9 pgs. [cited by applicant]
“International Application Serial No. PCT/US2024/048934, International Search Report mailed Dec. 18, 2024”, 5 pgs. [cited by applicant]
“International Application Serial No. PCT/US2024/048934, Written Opinion mailed Dec. 18, 2024”, 8 pgs. [cited by applicant]
“U.S. Appl. No. 17/489,394, Non Final Office Action mailed Feb. 7, 2025”, 42 pgs. [cited by applicant]
Cited By (4)
US 12,475,202 US 12,555,111 US 12,618,723 US 12,704,901