IP Library › Granted Patent US 12,585,335
Granted Patent B2
US 12,585,335 · App. 18/756,987 · Granted Mar 24, 2026

Gesture input with multiple views, displays and physics

Inventor: Glen J. Anderson (Portland, OR)
Assignee: Intel Corporation
G06F3/017G06F3/01G06F3/011G06F3/0304G06F3/0481G06F3/0484G06F3/14G06F3/1423
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,585,335
App. No.
18/756,987
Granted
Mar 24, 2026
Kind
B2
Abstract

Gesture input with multiple displays, views, and physics is described. In one example, a method includes generating a three dimensional space having a plurality of objects in different positions relative to a user and a virtual object to be manipulated by the user, presenting, on a display, a displayed area having at least a portion of the plurality of different objects, detecting an air gesture of the user against the virtual object, the virtual object being outside the displayed area, generating a trajectory of the virtual object in the three-dimensional space based on the air gesture, the trajectory including interactions with objects of the plurality of objects in the three-dimensional space, and presenting a portion of the generated trajectory on the displayed area.

Claims (34)

1 . An apparatus comprising:

interface circuitry;

machine-readable instructions; and

at least one processor circuit to be programmed by the instructions to:

identify an air gesture of a user;

determine a trajectory of a first virtual object through a three-dimensional space based on the air gesture, a first part of the trajectory within a first portion of the three-dimensional space that is not shown on a screen, a second part of the trajectory within a second portion of the three-dimensional space that is shown on a screen;

alter the second part of the trajectory based on interaction of the first virtual object with a second virtual object within the first portion of the three-dimensional space that is not shown on the screen; and

cause presentation of the first virtual object on the screen as the first virtual object moves along the second part of the trajectory.

2 . The apparatus of claim 1 , wherein the second part of the trajectory of the first virtual object is affected by a third virtual object within the second portion of the three-dimensional space that is shown on a screen.

3 . The apparatus of claim 1 , wherein the interaction includes the first virtual object bouncing off the second virtual object while the first virtual object is moving along the first part of the trajectory that is not shown on the screen.

4 . The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to cause presentation of the second virtual object based on the screen being rotated towards a location of the second virtual object within the three-dimensional space.

5 . The apparatus of claim 1 , wherein the first virtual object is to move along the first part of the trajectory before the second part of the trajectory, and one or more of the at least one processor circuit is to cause delay of the presentation of the first virtual object on the screen following the air gesture based on a length of the first part of the trajectory and a velocity of the first virtual object along the first part of the trajectory.

6 . The apparatus of claim 5 , wherein the length of the first part of the trajectory is based on a distance of the user from the screen.

7 . The apparatus of claim 6 , wherein the velocity of the first virtual object along the first part of the trajectory is based on at least one of (i) the air gesture or (ii) the interaction between the first virtual object and the second virtual object.

8 . The apparatus of claim 1 , wherein one or more of the at least one processor circuit is to define the three-dimensional space relative to the screen and relative to the user, the first portion of the three-dimensional space including a first area between the screen and the user, the second portion of the three-dimensional space including a second area beyond the screen relative to the user.

9 . At least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory comprising instructions to cause at least one processor circuit to at least:

identify an air gesture of a user;

determine a trajectory of a first virtual object through a three-dimensional space based on the air gesture, a first part of the trajectory within a first portion of the three-dimensional space that is not to be shown on a screen, a second part of the trajectory within a second portion of the three-dimensional space that is to be shown on a screen;

alter the second part of the trajectory based on interaction of the first virtual object with a second virtual object within the first portion of the three-dimensional space that is not to be shown on the screen; and

cause presentation of the first virtual object on the screen as the first virtual object moves along the second part of the trajectory.

10 . The at least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory of claim 9 , wherein the second part of the trajectory of the first virtual object is affected by a third virtual object within the second portion of the three-dimensional space that is to be shown on a screen.

11 . The at least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory of claim 9 , wherein the interaction includes the first virtual object bouncing off the second virtual object while the first virtual object is moving along the first part of the trajectory.

12 . The at least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory of claim 9 , wherein the instructions are to cause one or more of the at least one processor circuit to cause presentation of the second virtual object based on the screen being rotated towards a location of the second virtual object within the three-dimensional space.

13 . The at least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory of claim 9 , wherein the first virtual object is to move along the first part of the trajectory before the second part of the trajectory, and the instructions are to cause one or more of the at least one processor circuit to cause delay of the presentation of the first virtual object on the screen following the air gesture based on a length of the first part of the trajectory and a velocity of the first virtual object along the first part of the trajectory.

14 . The at least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory of claim 13 , wherein the length of the first part of the trajectory is based on a distance of the user from the screen.

15 . The at least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory of claim 14 , wherein the velocity of the first virtual object along the first part of the trajectory is based on at least one of (i) the air gesture or (ii) interaction between the first virtual object and the second virtual object.

16 . The at least one of an optical disk, an optical card, a magnetic-optical disk, random access memory (RAM), read-only memory (ROM), flash memory, or magnetic memory of claim 9 , wherein the instructions are to cause one or more of the at least one processor circuit to define the three-dimensional space relative to the screen and relative to the user, the first portion of the three-dimensional space including a first area between the screen and the user, the second portion of the three-dimensional space including a second area beyond the screen relative to the user.

17 . An apparatus comprising:

means for identifying an air gesture of a user;

means for determining a trajectory of a first virtual object through a three-dimensional space based on the air gesture, a first part of the trajectory within a first portion of the three-dimensional space that is not presentable to the user, a second part of the trajectory within a second portion of the three-dimensional space that is presentable to the user, the means for determining to alter the second part of the trajectory based on interaction of the first virtual object with a second virtual object within the first portion of the three-dimensional space that is not presentable to the user; and

means for presenting the first virtual object as the first virtual object moves along the second part of the trajectory.

18 . The apparatus of claim 17 , wherein the interaction includes the first virtual object bouncing off the second virtual object while the first virtual object is moving along the first part of the trajectory.

19 . The apparatus of claim 17 , wherein the means for determining is to cause presentation of the second virtual object based on the means for presenting being rotated towards a location of the second virtual object within the three-dimensional space.

20 . The apparatus of claim 17 , wherein the first virtual object is to move along the first part of the trajectory before the second part of the trajectory, and the means for determining is to cause delay of presentation of the first virtual object on the means for presenting following the air gesture based on a length of the first part of the trajectory and a velocity of the first virtual object along the first part of the trajectory.

Continuity (5)
Continuation 18146235 · Dec 23, 2022
Continuation 17215706 · Mar 29, 2021
Continuation 14954861 · Nov 30, 2015
Continuation 13977654
Related Publication 20240402828A1 · Dec 5, 2024
References Cited (88)
US 7532196B2 · Hinckley · 2009 [cited by applicant]
US 7665041B2 · Wilson et al. · 2010 [cited by applicant]
US 8072470B2 · Marks · 2011 [cited by applicant]
US 9557819B2 · Anderson · 2017 [cited by applicant]
US 10963062B2 · Anderson · 2021 [cited by applicant]
US 11543891B2 · Anderson · 2023 [cited by applicant]
US 12061745B2 · Anderson · 2024 [cited by applicant]
US 20020041327A1 · Hildreth et al. · 2002 [cited by applicant]
US 20040189720A1 · Wilson et al. · 2004 [cited by applicant]
US 20040193413A1 · Wilson et al. · 2004 [cited by applicant]
US 20040239670A1 · Marks · 2004 [cited by applicant]
US 20050093868A1 · Hinckley · 2005 [cited by applicant]
US 20050210418A1 · Marvit et al. · 2005 [cited by applicant]
US 20050245302A1 · Bathiche et al. · 2005 [cited by applicant]
US 20060192852A1 · Rosenthal · 2006 [cited by examiner]
US 20080018595A1 · Hildreth et al. · 2008 [cited by applicant]
US 20080030460A1 · Hildreth et al. · 2008 [cited by applicant]
US 20090268945A1 · Wilson et al. · 2009 [cited by applicant]
US 20100138798A1 · Wilson et al. · 2010 [cited by applicant]
US 20100146455A1 · Wilson et al. · 2010 [cited by applicant]
US 20100146464A1 · Wilson et al. · 2010 [cited by applicant]
US 20100151946A1 · Wilson et al. · 2010 [cited by applicant]
US 20100156906A1 · Montgomery · 2010 [cited by examiner]
US 20100195869A1 · Geiss · 2010 [cited by applicant]
US 20100197391A1 · Geiss · 2010 [cited by applicant]
US 20100197392A1 · Geiss · 2010 [cited by applicant]
US 20100197393A1 · Geiss · 2010 [cited by applicant]
US 20100197395A1 · Geiss · 2010 [cited by applicant]
US 20100197399A1 · Geiss · 2010 [cited by applicant]
US 20100197400A1 · Geiss · 2010 [cited by applicant]
US 20100241998A1 · Latta et al. · 2010 [cited by applicant]
US 20110173574A1 · Clavin et al. · 2011 [cited by applicant]
US 20110175932A1 · Yu et al. · 2011 [cited by applicant]
US 20110199291A1 · Tossell et al. · 2011 [cited by applicant]
US 20110289456A1 · Reville et al. · 2011 [cited by applicant]
US 20120038550A1 · Lemmey et al. · 2012 [cited by applicant]
US 20120084680A1 · Gimpl et al. · 2012 [cited by applicant]
US 20120157203A1 · Latta et al. · 2012 [cited by applicant]
US 20120206339A1 · Dahl · 2012 [cited by applicant]
US 20120236025A1 · Jacobsen et al. · 2012 [cited by applicant]
US 20120249429A1 · Anderson et al. · 2012 [cited by applicant]
US 20120257035A1 · Larsen · 2012 [cited by applicant]
US 20120295708A1 · Hernandez-Abrego et al. · 2012 [cited by applicant]
US 20130044128A1 · Liu · 2013 [cited by examiner]
US 20130067388A1 · Celie et al. · 2013 [cited by applicant]
US 20130147790A1 · Hildreth et al. · 2013 [cited by applicant]
US 20140092013A1 · Hildreth et al. · 2014 [cited by applicant]
US 20160179209A1 · Anderson · 2016 [cited by applicant]
US 20210286437A1 · Anderson · 2021 [cited by applicant]
CN 102184020A · 2011 [cited by applicant]
KR 20110117114A · 2011 [cited by applicant]
WO 2011004135A1 · 2011 [cited by applicant]
Gebhardt, “Superimposed Displays,” Medieninformatik Hauptseminar Sommersemester 2009 “Interactive Surfaces”, LFE Medieninformatik, retrieved from https://www.medien.ifi.Imu.de/lehre/ss09/hs/presentations/02_sascha_gebha… [cited by applicant]
International Searching Authority, “International Search Report,” issued in connection with International Patent Application No. PCT/US2011/062140, mailed on Jul. 31, 2012, 3 pages. [cited by applicant]
International Searching Authority, “Written Opinion,” issued in connection with International Patent Application No. PCT/US2011/062140, mailed on Jul. 31, 2012, 6 pages. [cited by applicant]
International Searching Authority, “International Preliminary Report on Patentability,” issued in connection with International Patent Application No. PCT/US2011/062140, issued on May 27, 2014, 7 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 13/977,654, dated Jul. 6, 2015, 9 pages. [cited by applicant]
Korean Intellectual Property Office, “Request for the Submission of an Opinion,” issued in connection with Korean Patent Application No. 2014-7017149, dated Jul. 17, 2015, 7 pages. [English translation included]. [cited by applicant]
European Patent Office, “Partial European Search Report,” issued in connection with European Patent Application No. 11876258.2, dated Jul. 27, 2015, 6 pages. [cited by applicant]
Korean Intellectual Property Office, “Written Decision on Registration,” issued in connection with Korean Patent Application No. 2014-7017149, dated Jan. 27, 2016, 2 pages. [English translation included]. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 13/977,654, dated Feb. 24, 2016, 7 pages. [cited by applicant]
European Patent Office, “Extended European Search Report,” issued in connection with European Patent Application No. 11876258.2, dated Mar. 15, 2016, 20 pages. [cited by applicant]
Korean Intellectual Property Office, “Request for the Submission of an Opinion,” issued in connection with Korean Patent Application No. 2016-7007033, dated Apr. 12, 2016, 4 pages. [English translation included]. [cited by applicant]
China National Intellectual Property Administration, “The First Office Action and Search Report,” issued in connection with Chinese Patent Application No. 201180076283.2, dated Apr. 29, 2016, 11 pages. [English translat… [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 13/977,654, dated Sep. 12, 2016, 5 pages. [cited by applicant]
Korean Intellectual Property Office, “Request for the Submission of an Opinion,” issued in connection with Korean Patent Application No. 2016-7007033, dated Oct. 24, 2016, 5 pages. [English translation included]. [cited by applicant]
China National Intellectual Property Administration, “The Second Office Action,” issued in connection with Chinese Patent Application No. 201180076283.2, dated Dec. 8, 2016, 7 pages. [English translation included]. [cited by applicant]
Korean Intellectual Property Office, “Written Decision on Registration,” issued in connection with Korean Patent Application No. 2016-7007033, dated Apr. 18, 2017, 3 pages. [English translation included]. [cited by applicant]
China National Intellectual Property Administration, “The Third Office Action,” issued in connection with Chinese Patent Application No. 201180076283.2, dated Jun. 27, 2017, 7 pages. [English translation included]. [cited by applicant]
China National Intellectual Property Administration, “Notification of Grant of Patent Right for Invention,” issued in connection with Chinese Patent Application No. 201180076283.2, dated Nov. 1, 2017, 3 pages. [English … [cited by applicant]
China National Intellectual Property Administration, “The First Office Action and Search Report,” issued in connection with Chinese Patent Application No. 201511009413.7, dated Jan. 29, 2018, 10 pages. [English translat… [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/954,861, dated Apr. 5, 2018, 10 pages. [cited by applicant]
European Patent Office, “Communication Under Rule 71(3) EPC,” issued in connection with European Patent Application No. 11876258.2, dated May 17, 2018, 7 pages. [cited by applicant]
China National Intellectual Property Administration, “The Second Office Action,” issued in connection with Chinese Patent Application No. 201511009413.7, dated Sep. 13, 2018, 11 pages. [English translation included]. [cited by applicant]
European Patent Office, “Decision to Grant a European Patent Pursuant to Article 97(1) EPC,” issued in connection with European Patent Application No. 11876258.2, dated Oct. 5, 2018, 2 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action,” issued in connection with U.S. Appl. No. 14/954,861, dated Nov. 2, 2018, 10 pages. [cited by applicant]
China National Intellectual Property Administration, “The Third Office Action,” issued in connection with Chinese Patent Application No. 201511009413.7, dated Mar. 15, 2019, 7 pages. [English translation included]. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/954,861, dated May 16, 2019, 12 pages. [cited by applicant]
China National Intellectual Property Administration, “Notification of Grant of Patent Right for Invention,” issued in connection with Chinese Patent Application No. 201511009413.7, dated Jul. 1, 2019, 3 pages. [English … [cited by applicant]
United States Patent and Trademark Office, “Final Office Action,” issued in connection with U.S. Appl. No. 14/954,861, dated Sep. 19, 2019, 13 pages. [cited by applicant]
United States Patent and Trademark Office, “Advisory Action,” issued in connection with U.S. Appl. No. 14/954,861, dated Feb. 10, 2020, 4 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 14/954,861, dated Feb. 24, 2020, 15 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 14/954,861, dated Jul. 14, 2020, 7 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 14/954,861, dated Nov. 18, 2020, 7 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 17/215,706, dated Feb. 3, 2022, 29 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 17/215,706, dated Aug. 9, 2022, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 18/146,235, dated Nov. 9, 2023, 35 Pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 18/146,235, dated Apr. 17, 2024, 7 pages. [cited by applicant]