IP Library › Granted Patent US 12,436,621
Granted Patent B2
US 12,436,621 · App. 18/428,927 · Granted Oct 7, 2025

Mechanism to provide visual feedback regarding computing system command gestures

Inventors: Rajiv Mongia (Redwood City, CA); Achintya Bhowmik (Milpitas, CA); Mark Yahiro (Santa Clara, CA); Dana Krieger (Emeryville, CA); Ed Mangum (San Mateo, CA); Diana Povieng (San Francisco, CA)
Assignee: Intel Corporation
G06F3/017G06F3/011G06F3/0481G06F3/04812G06F3/0487G06F3/04895G06F2203/04806
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,436,621
App. No.
18/428,927
Granted
Oct 7, 2025
Kind
B2
Abstract

A mechanism to provide visual feedback regarding computing system command gestures. An embodiment of an apparatus includes a sensing element to sense a presence or movement of a user of the apparatus, a processor, wherein operation of the processor includes interpretation of command gestures of a user to provide input to the apparatus; and a display screen, the apparatus to display one or more icons on the display screen, the one or more icons being related to the operation of the apparatus. The apparatus is to display visual feedback for a user of the apparatus, visual feedback including a representation of one or both hands of the user while the one or both hands are within a sensing area for the sensing element.

Claims (41)

1. Memory comprising instructions to cause at least one processor circuit to:

access first data, the first data based on a first output of one or more sensors, the first output associated with a first hand movement by a user relative to first content presented on a display screen;

determine a first gesture based on the first data;

in response to the first gesture, cause display of a first selectable option and a second selectable option;

access second data, the second data based on a second output of the one or more sensors, the second output associated with a second hand movement by the user, the second hand movement associated with selection of at least one of the first selectable option or the second selectable option; and

output a command associated with the selection.

2. The memory of claim 1 , wherein the instructions are to cause one or more of the at least one processor circuit to cause display of illuminated content.

3. The memory of claim 2 , wherein the instructions are to cause one or more of the at least one processor circuit to cause a brightness of the illuminated content to fade over time.

4. The memory of claim 1 , wherein the second data is indicative of a second gesture corresponding to the second hand movement, the second gesture different than the first gesture.

5. The memory of claim 1 , wherein the first hand movement includes movement of a finger of a hand of the user.

6. The memory of claim 1 , wherein the first selectable option and the second selectable option include text.

7. The memory of claim 1 , wherein the first selectable option and the second selectable option are displayed as a list.

8. An electronic device comprising:

a display;

a sensor;

machine-readable instructions; and

at least one processor circuit to be programmed by the machine-readable instructions to:

determine a first gesture based on first data, the first data based on a first output of the sensor, the first output associated with a first hand movement by a user relative to first content presented on the display;

in response to the first gesture, cause presentation of a menu on the display, the menu including a first selectable option and a second selectable option;

identify selection of one of the first selectable option or the second selectable option based on second data, the second data based on a second output of the sensor, the second output associated with a second hand movement by the user; and

output a command associated with the selection.

9. The electronic device of claim 8 , wherein the electronic device is a wireless personal electronic device.

10. The electronic device of claim 8 , wherein the sensor includes a camera.

11. The electronic device of claim 8 , wherein one or more of the at least one processor circuit is to determine a second gesture based on the second data, the second gesture different than the first gesture.

12. The electronic device of claim 11 , wherein the second gesture is defined by two fingers of a hand of the user.

13. The electronic device of claim 8 , wherein the first hand movement includes movement of a finger of a hand of the user.

14. The electronic device of claim 8 , wherein the command is to cause presentation of additional content on the display.

15. An apparatus comprising:

means for sensing movement;

means for displaying content;

computer readable instructions;

means for processing the computer readable instructions to:

determine a first gesture based on first data, the first data based on a first output of the sensing means, the first output associated with a first finger movement by a user relative to first content presented on the displaying means;

in response to the first gesture, cause presentation of a first selectable option and a second selectable option on the displaying means;

determine a second gesture based on second data, the second gesture indicative of selection of one of the first selectable option or the second selectable option, the second data based on a second output of the sensing means, the second output associated with a second finger movement by the user; and

output a command after the second gesture.

16. The apparatus of claim 15 , wherein the means for processing is to cause presentation of illuminated content on the displaying means.

17. The apparatus of claim 16 , wherein the means for processing is to cause a brightness of the illuminated content to fade over time.

18. The apparatus of claim 15 , wherein the first gesture is different than the second gesture.

19. The apparatus of claim 15 , wherein the first selectable option and the second selectable option include text.

20. The apparatus of claim 15 , wherein the first selectable option and the second selectable option are presented in a menu.

Continuity (4)
Continuation 17836674 · Jun 9, 2022
Continuation 16506583 · Jul 9, 2019
Continuation 13997640
Related Publication 20240288947A1 · Aug 29, 2024
References Cited (163)
US 5581670A · Bier · 1996 [cited by applicant]
US 5594469A · Freeman · 1997 [cited by applicant]
US 6128003A · Smith · 2000 [cited by applicant]
US 6222465B1 · Kumar · 2001 [cited by applicant]
US 7030861B1 · Westerman · 2006 [cited by applicant]
US 7274803B1 · Sharma · 2007 [cited by applicant]
US 7519223B2 · Dehlin · 2009 [cited by applicant]
US 7598942B2 · Underkoffler · 2009 [cited by applicant]
US 7849421B2 · Yoo et al. · 2010 [cited by applicant]
US 7924271B2 · Christie · 2011 [cited by applicant]
US 8181123B2 · Stone-Perez · 2012 [cited by applicant]
US 8333661B2 · Ng · 2012 [cited by applicant]
US 8334842B2 · Markovic · 2012 [cited by applicant]
US 8448083B1 · Migos · 2013 [cited by applicant]
US 8693724B2 · Ahmed · 2014 [cited by applicant]
US 8830227B2 · Rais · 2014 [cited by applicant]
US 8982182B2 · Shpunt · 2015 [cited by applicant]
US 9002099B2 · Litvak · 2015 [cited by applicant]
US 9069386B2 · Ajika · 2015 [cited by applicant]
US 9098931B2 · Shpunt · 2015 [cited by applicant]
US 9171200B2 · Subramanian · 2015 [cited by applicant]
US 9189073B2 · Mongia · 2015 [cited by applicant]
US 9189825B2 · Im · 2015 [cited by applicant]
US 9430128B2 · Hayes · 2016 [cited by applicant]
US 9678574B2 · Mongia · 2017 [cited by applicant]
US 9684379B2 · Mongia · 2017 [cited by applicant]
US 10324535B2 · Mongia et al. · 2019 [cited by applicant]
US 10345911B2 · Mongia · 2019 [cited by applicant]
US 11360566B2 · Mongia et al. · 2022 [cited by applicant]
US 11941181B2 · Mongia et al. · 2024 [cited by applicant]
US 20020064382A1 · Hildreth · 2002 [cited by applicant]
US 20020126026A1 · Lee · 2002 [cited by examiner]
US 20030036732A1 · Marano-Ford · 2003 [cited by applicant]
US 20030076293A1 · Mattsson · 2003 [cited by applicant]
US 20030156756A1 · Gokturk · 2003 [cited by examiner]
US 20050003851A1 · Chrysochoos · 2005 [cited by applicant]
US 20050052427A1 · Wu · 2005 [cited by applicant]
US 20050269950A1 · Giraldo · 2005 [cited by applicant]
US 20060026521A1 · Hotelling · 2006 [cited by applicant]
US 20060031786A1 · Hillis · 2006 [cited by examiner]
US 20060036944A1 · Wilson · 2006 [cited by applicant]
US 20060132432A1 · Bell · 2006 [cited by applicant]
US 20060187196A1 · Underkoffler · 2006 [cited by applicant]
US 20060209021A1 · Yoo · 2006 [cited by applicant]
US 20070230929A1 · Niwa · 2007 [cited by applicant]
US 20080005703A1 · Radivojevic · 2008 [cited by applicant]
US 20080036732A1 · Wilson · 2008 [cited by applicant]
US 20080052643A1 · Ike · 2008 [cited by applicant]
US 20080120577A1 · Ma · 2008 [cited by applicant]
US 20080163131A1 · Hirai · 2008 [cited by applicant]
US 20080231926A1 · Klug · 2008 [cited by examiner]
US 20080244465A1 · Kongqiao · 2008 [cited by examiner]
US 20090103780A1 · Nishihara · 2009 [cited by examiner]
US 20090217211A1 · Hildreth · 2009 [cited by applicant]
US 20090228841A1 · Hildreth · 2009 [cited by examiner]
US 20100031202A1 · Morris · 2010 [cited by applicant]
US 20100031203A1 · Morris · 2010 [cited by applicant]
US 20100045705A1 · Vertegaal · 2010 [cited by applicant]
US 20100060722A1 · Bell · 2010 [cited by applicant]
US 20100199232A1 · Mistry · 2010 [cited by examiner]
US 20100251112A1 · Hinckley · 2010 [cited by applicant]
US 20100280988A1 · Underkoffler · 2010 [cited by applicant]
US 20100281436A1 · Kipman · 2010 [cited by applicant]
US 20100281437A1 · Stone-Perez · 2010 [cited by applicant]
US 20100303289A1 · Polzin · 2010 [cited by applicant]
US 20110083106A1 · Hamagishi · 2011 [cited by examiner]
US 20110102570A1 · Wilf · 2011 [cited by applicant]
US 20110115702A1 · Seaberg · 2011 [cited by applicant]
US 20110119640A1 · Berkes · 2011 [cited by applicant]
US 20110164029A1 · King · 2011 [cited by examiner]
US 20110175802A1 · Hsieh · 2011 [cited by applicant]
US 20110193778A1 · Lee · 2011 [cited by examiner]
US 20110197263A1 · Stinson, III · 2011 [cited by applicant]
US 20110219340A1 · Pathangay · 2011 [cited by applicant]
US 20110221666A1 · Newton · 2011 [cited by examiner]
US 20110221669A1 · Shams · 2011 [cited by applicant]
US 20110258537A1 · Rives · 2011 [cited by applicant]
US 20110283212A1 · Warner · 2011 [cited by applicant]
US 20110289455A1 · Reville · 2011 [cited by applicant]
US 20110296353A1 · Ahmed et al. · 2011 [cited by applicant]
US 20110298708A1 · Hsu · 2011 [cited by examiner]
US 20120051588A1 · McEldowney · 2012 [cited by applicant]
US 20120069168A1 · Huang · 2012 [cited by examiner]
US 20120084688A1 · Robert · 2012 [cited by applicant]
US 20120117514A1 · Kim · 2012 [cited by examiner]
US 20120119988A1 · Izumi · 2012 [cited by examiner]
US 20120169646A1 · Berkes · 2012 [cited by examiner]
US 20120169670A1 · Kim · 2012 [cited by applicant]
US 20120179970A1 · Hayes · 2012 [cited by examiner]
US 20120235903A1 · Im · 2012 [cited by applicant]
US 20120249741A1 · Maciocci · 2012 [cited by applicant]
US 20120327125A1 · Kutliroff · 2012 [cited by examiner]
US 20130106898A1 · Saint-Loubert-Bie · 2013 [cited by applicant]
US 20130154919A1 · Tan · 2013 [cited by applicant]
US 20130265220A1 · Fleischmann · 2013 [cited by examiner]
US 20140013279A1 · Mongia · 2014 [cited by applicant]
US 20140208275A1 · Mongia · 2014 [cited by applicant]
US 20160091982A1 · Mongia et al. · 2016 [cited by applicant]
US 20160179209A1 · Anderson · 2016 [cited by applicant]
US 20190087008A9 · Mongia et al. · 2019 [cited by applicant]
US 20200174574A1 · Mongia et al. · 2020 [cited by applicant]
CA 2768893A1 · 2011 [cited by applicant]
CA 2768893C · 2015 [cited by applicant]
TW 393629B · 2000 [cited by applicant]
TW M294069U · 2006 [cited by applicant]
TW 201037577A · 2010 [cited by applicant]
TW 201120681A · 2011 [cited by applicant]
TW 201128541A · 2011 [cited by applicant]
TW 201137766A · 2011 [cited by applicant]
WO 2011066343A2 · 2011 [cited by applicant]
WO 2013095671A1 · 2013 [cited by applicant]
WO 2013095677A1 · 2013 [cited by applicant]
WO 2013095678A1 · 2013 [cited by applicant]
WO 2013095679A1 · 2013 [cited by applicant]
Derene, “Microsoft Surface: Behind-the-scenes First Look,” Popular Mechanics, Jun. 30, 2007, retrieved from https://www.popularmechanics.com/technology/gadgets/a1690/4217348/#ixzz1qRCVkoRW on May 16, 2024, 9 pages. [cited by applicant]
Ysnimkim, “Interactive UI in Minority Report,” YouTube, Jun. 7, 2008, retrieved from https://www.youtube.com/watch?v=NwVBzx0LMNQ on May 16, 2024, 8 pages. [cited by applicant]
International Searching Authority, “International Search Report,” issued in connection with International Patent Application No. PCT/US2011/067289, dated Aug. 28, 2012, 3 pages. [cited by applicant]
International Searching Authority, “Written Opinion of the International Searching Authority,” issued in connection with International Patent Application No. PCT/US2011/067289, dated Aug. 28, 2012, 5 pages. [cited by applicant]
International Searching Authority, “International Preliminary Report on Patentability,” issued in connection with International Patent Application No. PCT/US2011/067289, dated Jul. 3, 2014, 7 pages. [cited by applicant]
Taiwan Intellectual Property Office, “Official Letter,” mailed in connection with Taiwanese Patent Application No. 101149340, dated Jul. 28, 2014, 13 pages. [English language translation included.]. [cited by applicant]
Taiwan Intellectual Property Office, “Office Action,” mailed in connection with Taiwanese Patent Application No. 101146399, dated Sep. 23, 2014, 19 pages. [English language translation included.]. [cited by applicant]
United States Patent and Trademark Office, “Non-final Office Action,” mailed in connection with U.S. Appl. No. 13/997,978, dated Dec. 5, 2014, 5 pages. [cited by applicant]
Taiwan Intellectual Property Office, “Office Action,” mailed in connection with Taiwanese Patent Application No. 101147201, dated Feb. 24, 2015, 15 pages. [English language translation included.]. [cited by applicant]
Taiwan Intellectual Property Office, “Official Letter,” mailed in connection with Taiwanese Patent Application No. 101145913, dated Mar. 2, 2015, 16 pages. [English language translation included.]. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Action,” issued in U.S. Appl. No. 13/997,640 on Mar. 27, 2015, 16 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” mailed in connection with U.S. Appl. No. 13/997,978, dated Mar. 30, 2015, 7 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-final Office Action,” mailed in connection with U.S. Appl. No. 13/997,639, dated Jun. 12, 2015, 14 pages. [cited by applicant]
Taiwan Intellectual Property Office, “Office Action,” mailed in connection with Taiwanese Patent Application No. 101147201, dated Jun. 16, 2015, 8 pages. [English language translation included.]. [cited by applicant]
United States Patent and Trademark Office, “Non-final Office Action,” mailed in connection with U.S. Appl. No. 13/997,634, dated Jun. 18, 2015, 11 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Action,” issued in connection with U.S. Appl. No. 13/997,640 on Jul. 21, 2015, 19 pages. [cited by applicant]
European Patent Office, “Extended European Search Report,” mailed in connection with European Patent Application No. 11878127.7, dated Jul. 22, 2015, 6 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-final Office Action,” mailed in connection with U.S. Appl. No. 13/997,639, dated Dec. 24, 2015, 10 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action,” mailed in connection with U.S. Appl. No. 13/997,634, dated Jan. 4, 2016, 13 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-final Office Action,” mailed in connection with U.S. Appl. No. 13/997,634, dated Apr. 19, 2016, 13 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Action,” issued in connection with U.S. Appl. No. 13/997,640 on Apr. 22, 2016, 13 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Office Action,” mailed in connection with U.S. Appl. No. 13/997,639, dated May 16, 2016, 12 pages. [cited by applicant]
Taiwan Intellectual Property Office, “English Translation of Search Report,” mailed in connection with Taiwanese Patent Application No. 101147201, dated Jun. 21, 2016, 1 page. [cited by applicant]
United States Patent and Trademark Office, “Final Action,” issued in connection with U.S. Appl. No. 13/997,640, Aug. 10, 2016, 13 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” mailed in connection with U.S. Appl. No. 13/997,634, dated Aug. 16, 2016, 5 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” mailed in connection with U.S. Appl. No. 13/997,639, dated Nov. 8, 2016, 12 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” mailed in connection with U.S. Appl. No. 13/997,639, dated Feb. 16, 2017, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” mailed in connection with U.S. Appl. No. 13/997,634, dated Mar. 1, 2017, 5 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Action” issued in U.S. Appl. No. 13/997,640 on Sep. 7, 2017, 13 pages. [cited by applicant]
Taiwan Intellectual Property Office, “Official Letter,” mailed in connection with Taiwanese Patent Application No. 105122067, dated Jan. 8, 2018, 6 pages. [cited by applicant]
Taiwan Intellectual Property Office, “Official Letter,” mailed in connection with Taiwanese Patent Application No. 101145913, dated Jan. 8, 2018, 7 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Action” issued in connection with U.S. Appl. No. 14/954,858, Mar. 26, 2018, 9 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Action,” issued in connection with U.S. Appl. No. 13/997,640, Mar. 30, 2018, 13 pages. [cited by applicant]
Taiwan Intellectual Property Office, “Official Letter,” mailed in connection with Taiwanese Patent Application No. 105122067, dated Jul. 20, 2018, 10 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance” issued in connection with U.S. Appl. No. 14/954,858, Feb. 6, 2019, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” issued in connection with U.S. Appl. No. 13/997,640, Feb. 27, 2019, 5 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Action,” issued in connection with U.S. Appl. No. 16/506,583 on May 15, 2020, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Final Action,” issued in connection with U.S. Appl. No. 16/506,583, Feb. 19, 2021, 13 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Action,” issued in connection with U.S. Appl. No. 16/506,583, Jul. 15, 2021, 12 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” issued in connection with U.S. Appl. No. 16/506,583, Feb. 18, 2022, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance,” issued in connection with U.S. Appl. No. 16/506,583, Mar. 4, 2022, 5 pages. [cited by applicant]
International Searching Authority, “International Search Report,” mailed in connection with International Patent Application No. PCT/US2011/067290, dated Aug. 28, 2012, 3 pages. [cited by applicant]
International Searching Authority, “Written Opinion of the International Searching Authority,” mailed in connection with International Patent Application No. PCT/US2011/067290, dated Aug. 28, 2012, 5 pages. [cited by applicant]
International Searching Authority, “International Search Report,” mailed in connection with International Patent Application No. PCT/US2011/067287, dated Aug. 28, 2012, 3 pages. [cited by applicant]
International Searching Authority, “Written Opinion of the International Searching Authority,” mailed in connection with International Patent Application No. PCT/US2011/067287, dated Aug. 28, 2012, 5 pages. [cited by applicant]
International Searching Authority, “International Search Report,” mailed in connection with International Patent Application No. PCT/US2011/067280, dated Aug. 28, 2012, 3 pages. [cited by applicant]
International Searching Authority, “Written Opinion of the International Searching Authority,” mailed in connection with International Patent Application No. PCT/US2011/067280, dated Aug. 28, 2012, 5 pages. [cited by applicant]
United States Patent and Trademark Office, “Notice of Allowance and Fee(s) Due,” issued in connection with U.S. Appl. No. 17/836,674, dated Nov. 1, 2023, 8 pages. [cited by applicant]
United States Patent and Trademark Office, “Non-Final Office Action,” issued in connection with U.S. Appl. No. 17/836,674, mailed on May 10, 2023, 13 pages. [cited by applicant]