IP Library › Granted Patent US 12,379,821
Granted Patent B2
US 12,379,821 · App. 18/509,766 · Granted Aug 5, 2025

Smart assist for repeated actions

Inventors: Victor Carbune (Zurich, CH); Daniel Keysers (Stallikon, CH); Thomas Deselaers (Zurich, CH)
Assignee: GOOGLE LLC
G06F3/04817G06F3/0481G06F3/0482G06F9/451G06Q30/0601G06Q50/12G06V40/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,379,821
App. No.
18/509,766
Granted
Aug 5, 2025
Kind
B2
Abstract

Systems and methods enable a computing system to recognize a sequence of repeated actions and offer to automatically repeat any such recognized actions. An example method includes determining a current sequence of user actions is similar to previous sequence of user actions, determining whether the previous sequence is reproducible and, when reproducible, initiating display of a prompt that requests approval for completing the current sequence based on the previous sequence and, responsive to receiving an indication of approval, completing the previous sequence. Another example method included determining that a first current sequence of user interactions is complete and is not similar to any saved sequence of user interactions, saving the first current sequence as a previous sequence, identifying a second sequence as satisfying a similarity threshold with the previous sequence, and initiating display of a prompt that requests approval for saving the previous sequence as a shortcut.

Claims (90)

1. A method implemented by one or more processors comprising:

identifying a current sequence of user actions of a user with a mobile application executing at least in part at a computing device of the user;

identifying, based on the current sequence of user actions, a previous sequence of user actions of the user with the mobile application executing at least in part at the computing device of the user;

determining a confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions;

in response to determining that the confidence score fails to satisfy a first confidence threshold and a second confidence threshold:

classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as low; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as low:

refraining from storing the current sequence of actions; and

refraining from prompting the user to utilize the previous sequence of user actions to complete the current sequence of user actions;

in response to determining that the confidence score fails to satisfy the first confidence threshold but satisfies the second confidence threshold:

classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as medium; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as medium:

storing the current sequence of user actions in association with the previous sequence of user actions; and

refraining from prompting the user to utilize the previous sequence of user actions to complete the current sequence of user actions; and

in response to determining that the confidence score satisfies the first confidence threshold:

classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as high; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as high:

prompting, via the computing device of the user, the user to utilize the previous sequence of user actions to complete the current sequence of user actions.

2. The method of claim 1 , wherein determining the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions comprises:

processing, using a similarity prediction model, the previous sequence of user actions and the current sequence of user actions to generate a similarity metric between the previous sequence of user actions and the current sequence of user actions; and

utilizing the similarity metric as the confidence score.

3. The method of claim 2 , wherein the similarity metric indicates an extent of which it is predicted that the current sequence of user actions is repeating the previous sequence of user actions.

4. The method of claim 2 , wherein processing the previous sequence of user actions and the current sequence of user actions to generate the similarity metric between the previous sequence of user actions and the current sequence of user actions and using the similarity prediction model further comprises:

processing, using the similarity prediction model, and in addition to the previous sequence of user actions and the current sequence of user actions, metadata associated with the mobile application to generate the similarity metric between the previous sequence of user actions and the current sequence of user actions.

5. The method of claim 1 , wherein determining the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions comprises:

processing, using an action prediction model, the previous sequence of user actions and the current sequence of user actions to generate the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions.

6. The method of claim 5 , wherein processing the previous sequence of user actions and the current sequence of user actions to generate the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions and using the action prediction model further comprises:

processing, using the action prediction model, and in addition to the previous sequence of user actions and the current sequence of user actions, one or more contextual signals associated with the mobile application to generate the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions.

7. The method of claim 6 , wherein the one or more contextual signals comprise one or more of: behavior of the user across multiple sequences of user action, similar sequences of user actions of additional users that are in addition to the user, or whether a user interface of the mobile application has changed between the previous sequence of user actions being performed and the current sequence of user actions being performed.

8. The method of claim 1 , further comprising:

subsequent to prompting, via the computing device of the user, the user to utilize the previous sequence of user actions to complete the current sequence of user actions:

receiving user input to utilize the previous sequence of user actions to complete the current sequence of user actions; and

causing the previous sequence of user actions to be utilized to complete the current sequence of user actions.

9. The method of claim 8 , further comprising:

receiving further user input to store the previous sequence of user actions as a shortcut sequence of user actions.

10. A system comprising:

at least one processor; and

memory storing instructions that, when executed, cause the at least one processor to be operable to:

identify a current sequence of user actions of a user with a mobile application executing at least in part at a computing device of the user;

identify, based on the current sequence of user actions, a previous sequence of user actions of the user with the mobile application executing at least in part at the computing device of the user;

determine a confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions;

in response to determining that the confidence score fails to satisfy a first confidence threshold and a second confidence threshold:

classify the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as low; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as low:

refrain from storing the current sequence of actions; and

refrain from prompting the user to utilize the previous sequence of user actions to complete the current sequence of user actions;

in response to determining that the confidence score fails to satisfy the first confidence threshold but satisfies the second confidence threshold:

classify the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as medium; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as medium:

store the current sequence of user actions in association with the previous sequence of user actions; and

refrain from prompting the user to utilize the previous sequence of user actions to complete the current sequence of user actions; and

in response to determining that the confidence score satisfies the first confidence threshold:

classify the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as high; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as high:

prompt the user to utilize the previous sequence of user actions to complete the current sequence of user actions.

11. The system of claim 10 , wherein, in determining the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions, the at least one processor is to:

process, using a similarity prediction model, the previous sequence of user actions and the current sequence of user actions to generate a similarity metric between the previous sequence of user actions and the current sequence of user actions; and

utilize the similarity metric as the confidence score.

12. The system of claim 11 , wherein the similarity metric indicates an extent of which it is predicted that the current sequence of user actions is repeating the previous sequence of user actions.

13. The system of claim 11 , wherein, in processing the previous sequence of user actions and the current sequence of user actions to generate the similarity metric between the previous sequence of user actions and the current sequence of user actions and using the similarity prediction model, the at least one processor is to:

process, using the similarity prediction model, and in addition to the previous sequence of user actions and the current sequence of user actions, metadata associated with the mobile application to generate the similarity metric between the previous sequence of user actions and the current sequence of user actions.

14. The system of claim 10 , wherein, in determining the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions, the at least one processor is to:

process, using an action prediction model, the previous sequence of user actions and the current sequence of user actions to generate the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions.

15. The system of claim 14 , wherein, in processing the previous sequence of user actions and the current sequence of user actions to generate the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions and using the action prediction model, the at least one processor is to:

process, using the action prediction model, and in addition to the previous sequence of user actions and the current sequence of user actions, one or more contextual signals associated with the mobile application to generate the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions.

16. The system of claim 15 , wherein the one or more contextual signals comprise one or more of: behavior of the user across multiple sequences of user action, similar sequences of user actions of additional users that are in addition to the user, or whether a user interface of the mobile application has changed between the previous sequence of user actions being performed and the current sequence of user actions being performed.

17. The system of claim 10 , wherein the at least one processor is further operable to:

subsequent to prompting, via the computing device of the user, the user to utilize the previous sequence of user actions to complete the current sequence of user actions:

receive user input to utilize the previous sequence of user actions to complete the current sequence of user actions; and

cause the previous sequence of user actions to be utilized to complete the current sequence of user actions.

18. The system of claim 17 , wherein the at least one processor is further operable to:

receive further user input to store the previous sequence of user actions as a shortcut sequence of user actions.

19. A non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor to be operable to perform operations, the operations comprising:

identifying a current sequence of user actions of a user with a mobile application executing at least in part at a computing device of the user;

identifying, based on the current sequence of user actions, a previous sequence of user actions of the user with the mobile application executing at least in part at the computing device of the user;

determining a confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions;

in response to determining that the confidence score fails to satisfy a first confidence threshold and a second confidence threshold:

classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as low; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as low:

refraining from storing the current sequence of actions; and

refraining from prompting the user to utilize the previous sequence of user actions to complete the current sequence of user actions;

in response to determining that the confidence score fails to satisfy the first confidence threshold but satisfies the second confidence threshold:

classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as medium; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as medium:

storing the current sequence of user actions in association with the previous sequence of user actions; and

refraining from prompting the user to utilize the previous sequence of user actions to complete the current sequence of user actions; and

in response to determining that the confidence score satisfies the first confidence threshold:

classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as high; and

based on classifying the confidence score indicative of whether the previous sequence of user actions can be utilized to complete the current sequence of user actions as high:

prompting the user to utilize the previous sequence of user actions to complete the current sequence of user actions.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 11, 2023
From: CARBUNE, VICTOR; KEYSERS, DANIEL; DESELAERS, THOMAS
To: GOOGLE INC.
Reel/Frame 065829/0663 →
CHANGE OF NAME Recorded Dec 11, 2023
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 065854/0649 →
Continuity (3)
Continuation 17581390 · Jan 21, 2022
Continuation 15383966 · Dec 19, 2016
Related Publication 20240152249A1 · May 9, 2024
References Cited (215)
US 5731844A · Rauch et al. · 1998 [cited by applicant]
US 5946647A · Miller et al. · 1999 [cited by applicant]
US 6662226B1 · Wang et al. · 2003 [cited by applicant]
US 7054917B1 · Kirsch et al. · 2006 [cited by applicant]
US 7421153B1 · Ronca et al. · 2008 [cited by applicant]
US 7548915B2 · Ramer et al. · 2009 [cited by applicant]
US 7822759B2 · MacLaurin et al. · 2010 [cited by applicant]
US 7917514B2 · Lawler et al. · 2011 [cited by applicant]
US 8090222B1 · Baluja et al. · 2012 [cited by applicant]
US 8131786B1 · Bengio et al. · 2012 [cited by applicant]
US 8150911B2 · Bell et al. · 2012 [cited by applicant]
US 8204966B1 · Mendis et al. · 2012 [cited by applicant]
US 8255386B1 · Annau et al. · 2012 [cited by applicant]
US 8280414B1 · Nourse et al. · 2012 [cited by applicant]
US 8341156B1 · Westgate et al. · 2012 [cited by applicant]
US 8347237B2 · Bier et al. · 2013 [cited by applicant]
US 8418055B2 · King et al. · 2013 [cited by applicant]
US 8452799B2 · Zhou et al. · 2013 [cited by applicant]
US 8453058B1 · Coccaro et al. · 2013 [cited by applicant]
US 8468110B1 · Podgomy et al. · 2013 [cited by applicant]
US 8571319B2 · Balasubramanian et al. · 2013 [cited by applicant]
US 8645325B2 · Anderson et al. · 2014 [cited by applicant]
US 8671341B1 · Hellwig et al. · 2014 [cited by applicant]
US 8700604B2 · Roseman et al. · 2014 [cited by applicant]
US 8767497B2 · Marumoto et al. · 2014 [cited by applicant]
US 8799061B1 · Chatterjee et al. · 2014 [cited by applicant]
US 8954836B1 · Look et al. · 2015 [cited by applicant]
US 8977639B2 · Petrou et al. · 2015 [cited by applicant]
US 9081868B2 · Zhang et al. · 2015 [cited by applicant]
US 9165406B1 · Gray et al. · 2015 [cited by applicant]
US 9176984B2 · Hull et al. · 2015 [cited by applicant]
US 9276883B2 · Zhang et al. · 2016 [cited by applicant]
US 9354778B2 · Cornaby et al. · 2016 [cited by applicant]
US 9424668B1 · Petrou et al. · 2016 [cited by applicant]
US 9582482B1 · Sharifi et al. · 2017 [cited by applicant]
US 9703541B2 · Sharifi et al. · 2017 [cited by applicant]
US 11237696B2 · Carbune et al. · 2022 [cited by applicant]
US 20040117750A1 · Skoll et al. · 2004 [cited by applicant]
US 20050083413A1 · Reed et al. · 2005 [cited by applicant]
US 20060047639A1 · King et al. · 2006 [cited by applicant]
US 20060080594A1 · Chavoustie et al. · 2006 [cited by applicant]
US 20060106844A1 · Naick et al. · 2006 [cited by applicant]
US 20060141990A1 · Zak et al. · 2006 [cited by applicant]
US 20060156222A1 · Chi et al. · 2006 [cited by applicant]
US 20060221409A1 · Cohen et al. · 2006 [cited by applicant]
US 20060253491A1 · Gokturk et al. · 2006 [cited by applicant]
US 20070008321A1 · Gallagher et al. · 2007 [cited by applicant]
US 20070047781A1 · Hull et al. · 2007 [cited by applicant]
US 20070143345A1 · Jones et al. · 2007 [cited by applicant]
US 20070168379A1 · Patel et al. · 2007 [cited by applicant]
US 20070233671A1 · Oztekin et al. · 2007 [cited by applicant]
US 20080114604A1 · Wei et al. · 2008 [cited by applicant]
US 20080176606A1 · Kim · 2008 [cited by applicant]
US 20080235018A1 · Eggen et al. · 2008 [cited by applicant]
US 20080275701A1 · Wu et al. · 2008 [cited by applicant]
US 20080281974A1 · Slothouber et al. · 2008 [cited by applicant]
US 20080301101A1 · Baratto et al. · 2008 [cited by applicant]
US 20090005003A1 · Hartwell · 2009 [cited by applicant]
US 20090006388A1 · Ives et al. · 2009 [cited by applicant]
US 20090036215A1 · Saeki · 2009 [cited by applicant]
US 20090063431A1 · Erol et al. · 2009 [cited by applicant]
US 20090138466A1 · Henry et al. · 2009 [cited by applicant]
US 20090183124A1 · Sridhar · 2009 [cited by examiner]
US 20090228573A1 · Asakawa et al. · 2009 [cited by applicant]
US 20090228777A1 · Henry et al. · 2009 [cited by applicant]
US 20090252413A1 · Hua et al. · 2009 [cited by applicant]
US 20090282012A1 · Konig et al. · 2009 [cited by applicant]
US 20090319449A1 · Gamon et al. · 2009 [cited by applicant]
US 20100010987A1 · Smyth et al. · 2010 [cited by applicant]
US 20100060655A1 · Huang · 2010 [cited by applicant]
US 20100088612A1 · Jia et al. · 2010 [cited by applicant]
US 20100198768A1 · Zhou et al. · 2010 [cited by applicant]
US 20100250598A1 · Brauer et al. · 2010 [cited by applicant]
US 20100262928A1 · Abbott · 2010 [cited by applicant]
US 20100306249A1 · Hill et al. · 2010 [cited by applicant]
US 20100313141A1 · Yu et al. · 2010 [cited by applicant]
US 20110067025A1 · Cragun et al. · 2011 [cited by applicant]
US 20110072455A1 · Pickelsimer et al. · 2011 [cited by applicant]
US 20110107424A1 · Singh · 2011 [cited by applicant]
US 20110125735A1 · Petrou · 2011 [cited by applicant]
US 20110128288A1 · Petrou et al. · 2011 [cited by applicant]
US 20110131160A1 · Canny et al. · 2011 [cited by applicant]
US 20110131235A1 · Petrou et al. · 2011 [cited by applicant]
US 20110145692A1 · Noyes et al. · 2011 [cited by applicant]
US 20110191676A1 · Guttman et al. · 2011 [cited by applicant]
US 20110225152A1 · Beaudreau et al. · 2011 [cited by applicant]
US 20110238768A1 · Habets et al. · 2011 [cited by applicant]
US 20110246471A1 · Rakib · 2011 [cited by applicant]
US 20110258049A1 · Ramer et al. · 2011 [cited by applicant]
US 20110275358A1 · Faenger · 2011 [cited by applicant]
US 20110283296A1 · Chun · 2011 [cited by applicant]
US 20110307478A1 · Pinckney et al. · 2011 [cited by applicant]
US 20110307483A1 · Radlinski et al. · 2011 [cited by applicant]
US 20120044137A1 · Oddiraju et al. · 2012 [cited by applicant]
US 20120083294A1 · Bray et al. · 2012 [cited by applicant]
US 20120092286A1 · O'Prey et al. · 2012 [cited by applicant]
US 20120117058A1 · Rubinstein et al. · 2012 [cited by applicant]
US 20120158751A1 · Tseng · 2012 [cited by applicant]
US 20120159340A1 · Bae et al. · 2012 [cited by applicant]
US 20120191840A1 · Gordon · 2012 [cited by applicant]
US 20120194519A1 · Bissell et al. · 2012 [cited by applicant]
US 20120216102A1 · Malla · 2012 [cited by applicant]
US 20130080447A1 · Ramer et al. · 2013 [cited by applicant]
US 20130091463A1 · Nordstrom et al. · 2013 [cited by applicant]
US 20130097507A1 · Prewett · 2013 [cited by applicant]
US 20130108161A1 · Carr · 2013 [cited by applicant]
US 20130111328A1 · Khanna et al. · 2013 [cited by applicant]
US 20130117252A1 · Samaddar et al. · 2013 [cited by applicant]
US 20130173604A1 · Li et al. · 2013 [cited by applicant]
US 20130254217A1 · Xu · 2013 [cited by applicant]
US 20130263098A1 · Duda et al. · 2013 [cited by applicant]
US 20130290110A1 · LuVogt et al. · 2013 [cited by applicant]
US 20130311411A1 · Senanayake et al. · 2013 [cited by applicant]
US 20130325844A1 · Plaisant · 2013 [cited by applicant]
US 20130346431A1 · Erol et al. · 2013 [cited by applicant]
US 20140029810A1 · Barr et al. · 2014 [cited by applicant]
US 20140040272A1 · Houghton · 2014 [cited by applicant]
US 20140046965A1 · Tian et al. · 2014 [cited by applicant]
US 20140082670A1 · Papish · 2014 [cited by applicant]
US 20140089626A1 · Schluessler et al. · 2014 [cited by applicant]
US 20140095419A1 · Gandhi · 2014 [cited by examiner]
US 20140118597A1 · Tabak et al. · 2014 [cited by applicant]
US 20140142922A1 · Liang et al. · 2014 [cited by applicant]
US 20140146200A1 · Scott et al. · 2014 [cited by applicant]
US 20140152655A1 · Johnston et al. · 2014 [cited by applicant]
US 20140157210A1 · Katz et al. · 2014 [cited by applicant]
US 20140188889A1 · Martens et al. · 2014 [cited by applicant]
US 20140208234A1 · Amit et al. · 2014 [cited by applicant]
US 20140250147A1 · Shapira et al. · 2014 [cited by applicant]
US 20140279013A1 · Chelly et al. · 2014 [cited by applicant]
US 20140282660A1 · Oztaskent et al. · 2014 [cited by applicant]
US 20140316890A1 · Kagan · 2014 [cited by applicant]
US 20140362108A1 · Aguera-Arcas · 2014 [cited by applicant]
US 20140366158A1 · Han et al. · 2014 [cited by applicant]
US 20150016700A1 · Drozdzal et al. · 2015 [cited by applicant]
US 20150019997A1 · Kim et al. · 2015 [cited by applicant]
US 20150095855A1 · Bai et al. · 2015 [cited by applicant]
US 20150100524A1 · Pantel et al. · 2015 [cited by applicant]
US 20150121219A1 · Baklanovs · 2015 [cited by applicant]
US 20150169701A1 · Stekkelpak et al. · 2015 [cited by applicant]
US 20150178786A1 · Claessens · 2015 [cited by applicant]
US 20150212695A1 · Nordstrom et al. · 2015 [cited by applicant]
US 20150339405A1 · Vora et al. · 2015 [cited by applicant]
US 20160055246A1 · Marcin et al. · 2016 [cited by applicant]
US 20160234624A1 · Riva et al. · 2016 [cited by applicant]
US 20160307210A1 · Agarwal et al. · 2016 [cited by applicant]
US 20160321052A1 · Sharifi et al. · 2016 [cited by applicant]
US 20160350673A1 · Kumar et al. · 2016 [cited by applicant]
US 20170031575A1 · Dotan-Cohen · 2017 [cited by examiner]
US 20170098159A1 · Sharifi et al. · 2017 [cited by applicant]
US 20170118576A1 · Sharifi et al. · 2017 [cited by applicant]
US 20170139879A1 · Sharifi et al. · 2017 [cited by applicant]
US 20170153784A1 · Feltham et al. · 2017 [cited by applicant]
US 20170289292A1 · Aming et al. · 2017 [cited by applicant]
US 20180143744A1 · Movsisyan et al. · 2018 [cited by applicant]
US 20220147227A1 · Carbune et al. · 2022 [cited by applicant]
CN 101075236 · 2007 [cited by applicant]
CN 101201827 · 2008 [cited by applicant]
CN 101221575 · 2008 [cited by applicant]
CN 101233477 · 2008 [cited by applicant]
CN 101587495 · 2009 [cited by applicant]
CN 101763357 · 2010 [cited by applicant]
CN 102098380 · 2011 [cited by applicant]
CN 104506575 · 2015 [cited by applicant]
EP 2306290 · 2011 [cited by applicant]
EP 2466921 · 2012 [cited by applicant]
EP 2728481 · 2014 [cited by applicant]
EP 2824558 · 2015 [cited by applicant]
JP 2003114876 · 2003 [cited by applicant]
JP 2006065921 · 2006 [cited by applicant]
JP 2008026939 · 2008 [cited by applicant]
JP 2008526101 · 2008 [cited by applicant]
JP 2013077329 · 2013 [cited by applicant]
JP 2013210952 · 2013 [cited by applicant]
JP 2016040666 · 2016 [cited by applicant]
KR 1020140009687 · 2014 [cited by applicant]
KR 1020160005045 · 2016 [cited by applicant]
WO 03088080 · 2003 [cited by applicant]
WO 2006071264 · 2006 [cited by applicant]
WO 2009054619 · 2009 [cited by applicant]
WO 2010088217 · 2010 [cited by applicant]
WO 2012075315 · 2012 [cited by applicant]
WO 2013173940 · 2013 [cited by applicant]
WO 2014105922 · 2014 [cited by applicant]
WO 2014146265 · 2014 [cited by applicant]
WO 2016064857 · 2016 [cited by applicant]
European Patent Office; Communication issued in Application No. 23190640.5; 9 pages; dated Oct. 10, 2023. [cited by applicant]
Intellectual Property India; Hearing Notice issued for Application No. 201927023493, 2 pages, dated May 25, 2023. [cited by applicant]
European Patent Office; Intention to Grant issued for Application No. 17884158.1, 48 pages, dated Mar. 27, 2023. [cited by applicant]
Japanese Patent Office; Notice of Reasons for Rejection issued in App. No. 2019-532963, 6 pages, dated Dec. 19, 2022. [cited by applicant]
European Patent Office: Communication under Rule 71(3) EPC issue in Application No. 17884158.1; 46 pages; dated Oct. 11, 2022. [cited by applicant]
Japanese Patent Office; Notice of Reasons for Rejection issued in App. No. 2019-532963, 27 pages, dated Aug. 22, 2022. [cited by applicant]
Japanese Patent Office; Pre-Appeal Report issued in Application No. 2022-003654, 5 pages, dated Apr. 25, 2022. [cited by applicant]
China Intellectual Property Administration; Decision of Rejection issued for Application No. 201710906409.3; 6 pages; dated Jan. 10, 2022. [cited by applicant]
Japanese Patent Office; Decision of Rejection issued in Application No. 2019-532963, 4 pages; dated Nov. 15, 2021. [cited by applicant]
China National Intellectual Property Administration; Notification of Second Office Action; issued in Application No. 201710906409.3; 21 pages; dated Nov. 3, 2021. [cited by applicant]
The Korean Intellectual Property Office; Notice of Allowance issued in Application No. 10-2019-7020752; 3 pages; dated Nov. 5, 2021. [cited by applicant]
Japanese Patent Office; Notice of Reasons for Rejection issued in Application No. 2019-532963, 11 pages; dated Aug. 2, 2021. [cited by applicant]
Great Britain Intellectual Property Office; Examination Report issued in Application No. 1715581; 11 pages; dated Jul. 15, 2021. [cited by applicant]
China National Intellectual Property Administration; Notification of First Office Action; issued in Application No. 201710906409.3; 22 pages; dated May 7, 2021. [cited by applicant]
The Korean Intellectual Property Office; Notice of Office Action issued in Application No. 10-2019-7020752: 6 pages; dated May 27, 2021. [cited by applicant]
Intellectual Property India; Office Action issue in Application No. 201927023493; 7 pages; dated Feb. 8, 2021. [cited by applicant]
European Patent Office; Communication issue in Application No. 17884158.1; 15 pages; dated Nov. 27, 2020. [cited by applicant]
The Korean Intellectual Property Office: Notice of Office Action issue in Application No. 1020197020752; 17 pages; dated Nov. 24, 2020. [cited by applicant]
European Patent Office: Supplemental Search Report issued for Application No. 17884158.1 dated Aug. 12, 2020. [cited by applicant]
Zhou, D., et al., “Optimizing User Interaction for Web-based Mobile Tasks,” 2010 10th Annual International Symposium on Applications and the Internet; IEEE Computer Society; pp. 68-76 2010. [cited by applicant]
United Kingdom Intellectual Property Office; Examination Report issued in Application No. 1715581.3 dated Mar. 27, 2018. [cited by applicant]
Intellectual Property Office of Singapore- International Searching Authority; Notification of Transmittal of the International Search Report and the Written Opinion of PCT Serial No. PCT/US2017/053455; dated Dec. 27, 20… [cited by applicant]
Collobert, et al., “Natural Language Processing (almost) from Scratch”, Journal of Machine Learning Research, vol. 12, Aug. 1, 2011, pp. 2493-2537. [cited by applicant]
Minkov, et al., “Extracting Personal Names from Email: Applying Named Entity Recognition to Informal Text”, Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Pro… [cited by applicant]
“The new IFTTT is here”, IFTTT Blog (https://ifttt.com/blog/2016/11/the-new-ifttt-is-here), printed Nov. 3, 2016, 6 pages. [cited by applicant]
Adistambha, et al., “Efficient Multimedia query-by-content from mobile devices”, Computers & Electrical Engineering XX, vol. 36, No. 4, 2010, pp. 626-642. [cited by applicant]
Hsu, et al., “Snap2Read: Automatic Magazine Capturing and Analysis for Adaptive Mobile Reading,” Lecture Notes in Computer Science, vol. 6524, 2011, 11 pages. [cited by applicant]
Japan Patent Office, Office Action issued in Application No. 2019-532963, dated Nov. 2, 2020 (13 pages). [cited by applicant]
Deutsches Patent Office; Office Action issued in Application No. 102017121758.4; 18 pages; dated Feb. 12, 2024. [cited by applicant]