Granted Patent B2

US 9,961,127 · App. 14/216,889 · Granted May 1, 2018

Abstract

Claims (29)

1. A computer-implemented method comprising the steps of:

embedding a tracking module in a host application that is implemented on a mobile device including an operating system that accesses a main user interface (UI) thread, wherein the tracking module comprises an application programming interface (API);

capturing with the tracking module interaction data relating to the host application wherein the interaction data comprises (i) visual interaction data including images presented on a display of the mobile device and (ii) non-visual interaction data including meta-data comprising at least one of touch-event data, gesture data, GPS coordinate data, audio data, accelerometer data, orientation data, camera data, barometer data, and compass data and wherein the tracking module captures the visual interaction data according to a swizzling technique by accessing the main user UI thread shared by the operating system and by intermittently accessing a second UI thread that is independent of the main UI thread thereby increasing availability of the main UI thread for the operating system during capture of the visual interaction data, and wherein the swizzling technique is a thread safe operation; and

transmitting the captured visual and non-visual interaction data from the mobile device to a tracking server.

2. The method of claim 1 wherein capturing the images includes capturing a change to the images wherein the change is initiated by a user.

3. The method of claim 1 wherein capturing the images includes capturing a change to the images wherein the change is initiated by the host application independent of a user.

4. The method of claim 1 wherein capturing the visual interaction data includes referencing user interface objects of the host application.

5. The method of claim 1 wherein capturing the interaction data includes capturing a time associated with the interaction data.

6. The method of claim 1 further including measuring with the tracking module a period of time required to capture the interaction data.

7. The method of claim 6 further including utilizing the tracking module to dynamically alter a frequency at which the interaction data is captured based on the measured period of time required to the capture interaction data.

8. The method of claim 1 further including capturing with the tracking module data representing the availability of resources of the mobile device.

9. The method of claim 8 further including utilizing the tracking module to dynamically alter a frequency at which the interaction data is captured based on the captured data representing the availability of resources of the mobile device.

10. The method of claim 1 further including storing the captured interaction data to a buffer.

11. The method of claim 10 further including retrieving with the tracking module the captured interaction data from the buffer.

12. The method of claim 1 further including processing the captured interaction data with the tracking module in preparation for transmission.

13. The method of claim 12 wherein processing the captured interaction data includes encoding captured interaction data.

14. The method of claim 13 wherein encoding the captured interaction data includes compressing the captured interaction data into a video.

15. The method of claim 13 wherein encoding the captured interaction data includes applying an image differencing process to capture visual differences between sequential interaction data.

16. The method of claim 1 wherein transmitting the captured interaction data includes transmitting the captured interaction data upon occurrence of a predetermined event including at least one of a lapse of a predetermined threshold of time, a detection of a predetermined network connection, a detection of a command or status of the mobile device, an acceptance of an invitation to complete a survey, and a completion of a survey.

17. The method of claim 1 wherein transmitting the captured interaction data is initiated by the tracking module.

18. The method of claim 1 further including utilizing the tracking module to mask portions of the captured interaction data before transmission.

19. The method of claim 1 further comprising:

receiving the captured interaction data on the tracking server;

converting the captured interaction data received on the tracking server into a visual representation;

requesting with an analysis computer the visual representation from the tracking server; and

presenting the requested visual representation on the analysis computer.

20. The method of claim 1 further comprising capturing the visual interaction data by:

acquiring, with the tracking module, layers of an object from a user interface of the host application; and

rendering layers of the object.

Assignments (13)