IP Library Granted Patent US 9,684,445
Granted Patent B2
US 9,684,445 · App. 15/072,430 · Granted Jun 20, 2017

Mobile gesture reporting and replay with unresponsive gestures identification and analysis

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,684,445
App. No.
15/072,430
Granted
Jun 20, 2017
Kind
B2
Abstract

In gesture reporting and replay, touch events are captured from an operating system of a client device for touch actions on a touchscreen. The touch events are interpreted as gestures. Any application events are captured from an application in response to the gestures. A given gesture and any given application event in response to the given gesture are compared against an expected application event. When there is a match, the given gesture is identified as a responsive gesture. When there is no match, the given gesture is identified as a non-responsive gesture. The gesture information for the given gesture, any application event in response to the given gesture, and a screen layout displayed on the touchscreen at the time of the given gesture are sent to a service. For the information received, the service replays the given gesture and generates a report.

Claims (46)

1. A method for gesture reporting and replay, comprising:

capturing one or more touch events from an operating system of a client device for touch actions that occur on a touchscreen coupled to the client device;

interpreting the one or more touch events as a gesture;

capturing an application event from an application, where the captured application event occurs in response to the gesture;

determining, in response to interpreting the one or more touch events as the gesture, an expected application event corresponding to the gesture;

comparing the captured application event against the expected application event;

when the captured application event matches the expected application event, identifying the gesture as a responsive gesture; and

when the captured application event does not match the expected application event, identifying the gesture as a non-responsive gesture.

2. The method of claim 1 , further comprising:

receiving data from the client device, where the data comprises gesture information for the gesture, the captured application event, and a screen layout displayed on the touchscreen at the time of the gesture;

parsing the data to identify one or more unresponsive events for the non-responsive gesture; and

storing the one or more unresponsive events.

3. The method of claim 2 , further comprising:

receiving a search request for unresponsive events;

in response to the search request, obtaining the one or more unresponsive events;

sending the one or more unresponsive events to a replay engine; and

rendering, by the replay engine, one or more views comprising the gesture, the captured application event, and the screen layout for the one or more unresponsive events, wherein the one or more views are replayed by a replay client device.

4. The method of claim 3 , further comprising:

receiving the one or more views at the replay client device; and

displaying the one or more views with the one or more unresponsive events highlighted.

5. The method of claim 2 , further comprising:

generating a report using the gesture information for the gesture, the captured application event in response to the gesture, and the screen layout displayed on the touchscreen at the time of the gesture.

6. The method of claim 5 , further comprising: generating the report for a plurality of application sessions across a plurality of client devices.

7. The method of claim 1 , wherein the determining of the expected application event comprises obtaining expected application event information, where the expected application event comprises an intended application response corresponding to the gesture, the screen layout at the time of the gesture, and an expected time lapse between the gesture and the captured application event in response to the gesture.

8. The method of claim 1 , wherein the expected application event is selected from a group consisting of an application event that logically follows the gesture and an application event specifically defined for the application.

9. The method of claim 1 , wherein the gesture interpreted from the one or more touch events is one gesture of a plurality of gestures, and the method comprises:

interpreting the plurality of gestures from the one or more touch events,

wherein the captured application event that occurs in response to the one gesture is also in response the other gestures of the plurality of gestures,

wherein the expected application event that is determined in response to interpreting the one or more touch events as the one gesture is also determined in response to interpreting the one or more touch events as the other gestures of the plurality of gestures, so that the expected application event corresponds to the plurality of gestures;

identifying the plurality of gestures as a plurality of responsive gestures when the captured application event matches the expected application event; and

identifying the plurality of gestures as a plurality of non-responsive gestures when the captured application event does not match the expected application event.

10. A method for gesture reporting and replay, implemented by a processor, comprising:

receiving, by the processor, data from a client device, where the data comprises gesture information for a gesture, a captured application event that occurred in response to the gesture, and a screen layout displayed on a touchscreen at the time of the gesture;

parsing the data to identify one or more unresponsive events for a non-responsive gesture, wherein the non-responsive gesture is identified by:

capturing one or more touch events from an operating system of the client device for touch actions that occur on the touchscreen coupled to the client device;

interpreting the one or more touch events as a gesture;

capturing the application event from an application, where the captured application event occurs in response to the gesture;

determining, in response to interpreting the one or more touch events as the gesture, an expected application event corresponding to the gesture;

comparing the captured application event against the expected application event;

when the captured application event matches the expected application event, identifying the gesture as a responsive gesture; and

when the captured application event does not match the expected application event, identifying the gesture as the non-responsive gesture;

storing the one or more unresponsive events for the non-responsive gesture;

receiving a search request for unresponsive events;

in response to the search request, obtaining the one or more unresponsive events;

sending the one or more unresponsive events to a replay engine; and

rendering, by the replay engine, one or more views comprising the gesture, the captured application event, and the screen layout displayed on the touchscreen at the time of the gesture for the one or more unresponsive events, wherein the one or more views are replayed by a replay client device.

Assignments (5)
TERMINATION AND RELEASE OF SECURITY INTEREST IN PATENTS Recorded Oct 19, 2022
From: GOLDMAN SACHS SPECIALTY LENDING GROUP, L.P.
To: ACOUSTIC, L.P.
Reel/Frame 061713/0942 →
SECURITY INTEREST Recorded Oct 19, 2022
From: ACOUSTIC, L.P.
To: ALTER DOMUS (US) LLC, AS COLLATERAL AGENT
Reel/Frame 061720/0361 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 5, 2019
From: INTERNATIONAL BUSINESS MACHINES CORPORATION
To: ACOUSTIC, L.P.
Reel/Frame 049964/0263 →
SECURITY INTEREST Recorded Jun 28, 2019
From: ACOUSTIC, L.P.
To: GOLDMAN SACHS SPECIALTY LENDING GROUP, L.P.
Reel/Frame 049629/0649 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 17, 2016
From: HERNANDEZ, OMAR; MADAN, VIKAS; SIMPSON, SOPHIA A.; SINGH, SUKHBIR
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 038008/0752 →