IP Library Granted Patent US 12,638,951
Granted Patent B2
US 12,638,951 · App. 18/437,004 · Granted May 26, 2026

3D user interface depth forgiveness

Inventor: Francis Patrick Sullivan (Sandy, UT)
Assignee: Snap Inc.
G06F3/04815G06T19/006G06V40/28
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,638,951
App. No.
18/437,004
Filed
Feb 8, 2024
Granted
May 26, 2026
Kind
B2
Art Unit
2179
USPC
715/848
Abstract

A head-worn device system includes one or more cameras, one or more display devices and one or more processors. The system also includes a memory storing instructions that, when executed by the one or more processors, configure the system to generate a virtual object, generate a virtual object collider for the virtual object, determine a conic collider for the virtual object, provide the virtual object to a user, detect a landmark on the user's hand in the real-world, generate a landmark collider for the landmark, and determine a selection of the first virtual object by the user based on detecting a collision between the landmark collider with the conic collider and with the virtual object collider.

Claims (46)

1 . A machine-implemented method comprising:

generating, by one or more processors, a virtual object in a 3D user interface;

generating, by the one or more processors, a first virtual object collider for the virtual object;

generating, by the one or more processors, a second virtual object collider for the virtual object wherein a radius of the second virtual object collider is greater than a radius of the first virtual object collider;

providing, by the one or more processors, the virtual object to a user in a display;

detecting, by the one or more processors, a landmark on a hand of the user;

generating, by the one or more processors, a landmark collider for the landmark;

generating, by the one or more processors, a ray originating at an eye position of the user and passing through the landmark; and

determining, by the one or more processors, a user interaction with the virtual object based on detecting an intersection between the landmark collider with the second virtual object collider and a collision of the ray with the first virtual object collider.

2 . The machine-implemented method of claim 1 wherein an axis of the first virtual object collider runs through a centroid of the virtual object.

3 . The machine-implemented method of claim 1 wherein an axis of the second virtual object collider runs through a centroid of the virtual object.

4 . The machine-implemented method of claim 1 wherein a radius of the first virtual object collider is equal to a radius of a sphere circumscribing the virtual object.

5 . The machine-implemented method of claim 1 wherein a radius of the second virtual object collider is greater than a radius of a sphere circumscribing the virtual object.

6 . The machine-implemented method of claim 1 , wherein the first virtual object collider is a sphere circumscribing the virtual object and the second virtual object collider is an expanded sphere having a radius larger than the first virtual object collider.

7 . The machine-implemented method of claim 1 wherein the virtual object is generated based on real-world video frame data.

8 . A machine comprising:

at least one processor; and

at least one memory storing instructions that, when executed by the at least one processor, cause the machine to perform operations comprising:

generate, by one or more processors, a virtual object in a 3D user interface;

generate, by the one or more processors, a first virtual object collider for the virtual object;

generate, by the one or more processors, a second virtual object collider for the virtual object wherein a radius of the second virtual object collider is greater than a radius of the first virtual object collider;

provide, by the one or more processors, the virtual object to a user in a display;

detect, by the one or more processors, a landmark on a hand of the user;

generate, by the one or more processors, a landmark collider for the landmark;

generate, by the one or more processors, a ray originating at an eye position of the user and passing through the landmark; and

determine, by the one or more processors, a user interaction with the virtual object based on detecting an intersection between the landmark collider with the second virtual object collider and a collision of the ray with the first virtual object collider.

9 . The computing apparatus of claim 8 wherein an axis of the first virtual object collider runs through a centroid of the virtual object.

10 . The computing apparatus of claim 8 wherein an axis of the second virtual object collider runs through a centroid of the virtual object.

11 . The computing apparatus of claim 8 wherein a radius of the first virtual object collider is equal to a radius of a sphere circumscribing the virtual object.

12 . The computing apparatus of claim 8 wherein a radius of the second virtual object collider is greater than a radius of a sphere circumscribing the virtual object.

13 . The computing apparatus of claim 8 , wherein the first virtual object collider is a sphere circumscribing the virtual object and the second virtual object collider is an expanded sphere having a radius larger than the first virtual object collider.

14 . The computing apparatus of claim 8 wherein the virtual object is generated based on real-world video frame data.

15 . A machine-storage medium storing instructions that, when executed by a machine, cause the machine to perform operations comprising:

generate, by one or more processors, a virtual object in a 3D user interface;

generate, by the one or more processors, a first virtual object collider for the virtual object;

generate, by the one or more processors, a second virtual object collider for the virtual object wherein a radius of the second virtual object collider is greater than a radius of the first virtual object collider;

provide, by the one or more processors, the virtual object to a user in a display;

detect, by the one or more processors, a landmark on a hand of the user;

generate, by the one or more processors, a landmark collider for the landmark;

generate, by the one or more processors, a ray originating at an eye position of the user and passing through the landmark; and

determine, by the one or more processors, a user interaction with the virtual object based on detecting an intersection between the landmark collider with the second virtual object collider and a collision of the ray with the first virtual object collider.

16 . The machine-storage medium of claim 15 wherein an axis of the first virtual object collider runs through a centroid of the virtual object.

17 . The machine-storage medium of claim 15 wherein an axis of the second virtual object collider runs through a centroid of the virtual object.

18 . The machine-storage medium of claim 15 wherein a radius of the first virtual object collider is equal to a radius of a sphere circumscribing the virtual object.

19 . The machine-storage medium of claim 15 wherein a radius of the second virtual object collider is greater than a radius of a sphere circumscribing the virtual object.

20 . The machine-storage medium of claim 15 , wherein the first virtual object collider is a sphere circumscribing the virtual object and the second virtual object collider is an expanded sphere having a radius larger than the first virtual object collider.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2024
From: SULLIVAN, FRANCIS PATRICK
To: SNAP INC.
Reel/Frame 066587/0428 →
Continuity (3)
Continuation 18099155 · Jan 19, 2023
Continuation 17694488 · Mar 14, 2022
Related Publication 20240184415A1 · Jun 6, 2024
References Cited (100)
US 7971156B2 · Albertson et al. · 2011 [cited by applicant]
US 7996793B2 · Latta et al. · 2011 [cited by applicant]
US 8487938B2 · Latta et al. · 2013 [cited by applicant]
US 8856691B2 · Geisner et al. · 2014 [cited by applicant]
US 9225897B1 · Sehn et al. · 2015 [cited by applicant]
US 9230160B1 · Kanter · 2016 [cited by applicant]
US 9276886B1 · Samaranayake · 2016 [cited by applicant]
US 9705831B2 · Spiegel · 2017 [cited by applicant]
US 9742713B2 · Spiegel et al. · 2017 [cited by applicant]
US 10102423B2 · Shaburov et al. · 2018 [cited by applicant]
US 10284508B1 · Allen et al. · 2019 [cited by applicant]
US 10439972B1 · Spiegel et al. · 2019 [cited by applicant]
US 10509466B1 · Miller et al. · 2019 [cited by applicant]
US 10514876B2 · Sehn · 2019 [cited by applicant]
US 10579869B1 · Xiong et al. · 2020 [cited by applicant]
US 10614855B2 · Huang · 2020 [cited by applicant]
US 10635161B2 · Clement et al. · 2020 [cited by applicant]
US 10748347B1 · Li et al. · 2020 [cited by applicant]
US 10958608B1 · Allen et al. · 2021 [cited by applicant]
US 10962809B1 · Castañeda · 2021 [cited by applicant]
US 10996846B2 · Robertson et al. · 2021 [cited by applicant]
US 10997787B2 · Ge et al. · 2021 [cited by applicant]
US 11012390B1 · Al Majid et al. · 2021 [cited by applicant]
US 11030454B1 · Xiong et al. · 2021 [cited by applicant]
US 11036368B1 · Al Majid et al. · 2021 [cited by applicant]
US 11062498B1 · Voss et al. · 2021 [cited by applicant]
US 11087728B1 · Canberk et al. · 2021 [cited by applicant]
US 11092998B1 · Castañeda et al. · 2021 [cited by applicant]
US 11106342B1 · Al Majid et al. · 2021 [cited by applicant]
US 11126206B2 · Meisenholder et al. · 2021 [cited by applicant]
US 11143867B2 · Rodriguez, II · 2021 [cited by applicant]
US 11151798B1 · Casaburo · 2021 [cited by examiner]
US 11164380B2 · Mchugh et al. · 2021 [cited by applicant]
US 11169600B1 · Canberk et al. · 2021 [cited by applicant]
US 11227626B1 · Krishnan Gorumkonda et al. · 2022 [cited by applicant]
US 11257280B1 · Pedrotti · 2022 [cited by examiner]
US 11307747B2 · Dancie et al. · 2022 [cited by applicant]
US 11531402B1 · Stolzenberg · 2022 [cited by applicant]
US 11546505B2 · Canberk · 2023 [cited by applicant]
US 11579747B1 · Sullivan · 2023 [cited by applicant]
US 20090012788A1 · Gilbert et al. · 2009 [cited by applicant]
US 20110301934A1 · Tardif · 2011 [cited by applicant]
US 20140171036A1 · Simmons · 2014 [cited by applicant]
US 20140204120A1 · Moteki et al. · 2014 [cited by applicant]
US 20150120293A1 · Wohlert et al. · 2015 [cited by applicant]
US 20150221132A1 · Kruglick · 2015 [cited by examiner]
US 20150370320A1 · Connor · 2015 [cited by applicant]
US 20170123487A1 · Hazra et al. · 2017 [cited by applicant]
US 20170277684A1 · Dharmarajan Mary · 2017 [cited by applicant]
US 20170277685A1 · Takumi · 2017 [cited by applicant]
US 20170329515A1 · Clement · 2017 [cited by examiner]
US 20170351910A1 · Elwazer et al. · 2017 [cited by applicant]
US 20180158370A1 · Pryor · 2018 [cited by applicant]
US 20180307303A1 · Powderly et al. · 2018 [cited by applicant]
US 20190155397A1 · Liu et al. · 2019 [cited by applicant]
US 20190295280A1 · Diverdi et al. · 2019 [cited by applicant]
US 20200043236A1 · Miller et al. · 2020 [cited by applicant]
US 20210011612A1 · Dancie et al. · 2021 [cited by applicant]
US 20210074016A1 · Li et al. · 2021 [cited by applicant]
US 20210166732A1 · Shaburova et al. · 2021 [cited by applicant]
US 20210174034A1 · Retek et al. · 2021 [cited by applicant]
US 20210241529A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210263593A1 · Lacey · 2021 [cited by applicant]
US 20210303075A1 · Cowburn et al. · 2021 [cited by applicant]
US 20210303077A1 · Anvaripour et al. · 2021 [cited by applicant]
US 20210303140A1 · Mourkogiannis · 2021 [cited by applicant]
US 20210382564A1 · Blachly et al. · 2021 [cited by applicant]
US 20210397000A1 · Rodriguez, II · 2021 [cited by applicant]
US 20210405761A1 · Canberk · 2021 [cited by applicant]
US 20220188539A1 · Chan et al. · 2022 [cited by applicant]
US 20220206588A1 · Canberk et al. · 2022 [cited by applicant]
US 20220300730A1 · Eirinberg et al. · 2022 [cited by applicant]
US 20220300731A1 · Eirinberg et al. · 2022 [cited by applicant]
US 20220326781A1 · Hwang et al. · 2022 [cited by applicant]
US 20220334649A1 · Hwang et al. · 2022 [cited by applicant]
US 20230289030A1 · Sullivan · 2023 [cited by applicant]
CN 103049761B · 2016 [cited by applicant]
EP 3707693A1 · 2020 [cited by applicant]
KR 20220057768A · 2022 [cited by applicant]
KR 20220158824A · 2022 [cited by applicant]
WO WO2016168591A1 · 2016 [cited by applicant]
WO WO2019094618A1 · 2019 [cited by applicant]
WO WO2022005687A1 · 2022 [cited by applicant]
WO WO2022005693A1 · 2022 [cited by applicant]
WO WO2022060549A2 · 2022 [cited by applicant]
WO WO2022066578A1 · 2022 [cited by applicant]
WO WO2022132381A1 · 2022 [cited by applicant]
WO WO2022146678A1 · 2022 [cited by applicant]
WO WO2022198182A1 · 2022 [cited by applicant]
WO WO2022216784A1 · 2022 [cited by applicant]
WO WO2022225761A1 · 2022 [cited by applicant]
WO WO2022245765A1 · 2022 [cited by applicant]
WO WO2023177621A1 · 2023 [cited by applicant]
“U.S. Appl. No. 17/694,488, Notice of Allowance mailed Oct. 13, 2022”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 18/099,155, Notice of Allowability mailed Jul. 26, 2023”, 6 pgs. [cited by applicant]
“U.S. Appl. No. 18/099,155, Notice of Allowance mailed Jun. 26, 2023”, 9 pgs. [cited by applicant]
“U.S. Appl. No. 18/099,155, Notice of Allowance mailed Nov. 8, 2023”, 8 pgs. [cited by applicant]
“International Application Serial No. PCT/US2023/015104, International Search Report mailed Jun. 2, 2023”, 4 pgs. [cited by applicant]
“International Application Serial No. PCT/US2023/015104, Written Opinion mailed Jun. 2, 2023”, 8 pgs. [cited by applicant]
“Korean Application Serial No. 10-2024-7033765, Notice of Preliminary Rejection mailed Dec. 18, 2025”, w / English 1 translation, 13 pgs. [cited by applicant]