Granted Patent B2

US 9,799,143 · App. 14/461,277 · Granted Oct 24, 2017

Inventors: Brian Mullins (Sierra Madre, CA); Matthew Kammerait (West Hollywood, CA); Frank Chester Irving, Jr. (Woodland Hills, CA)

Assignee: DAQRI, LLC

G06T19/006G06F3/011G06T11/206

Abstract

Claims (55)

1. A server comprising:

a hardware processor comprising an augmented reality application, the augmented reality application configured to perform operations comprising:

performing an analytics computation of users' interactions with an augmented reality content based on a physical object captured by a viewing device, the analytics computation comprising a computation of geometric paths of the users' interactions with the augmented reality content;

causing a display of a visualization of the analytics computation from the users' interactions with the augmented reality content;

accessing and processing path bundle data packages received from the viewing device, the path bundle data packages describing a user's interaction of the viewing device with the augmented reality content based on a physical object captured by the viewing device, the path bundle data packages generated based on sensor data using a data model comprising a data header and a data payload, the data header comprising a contextual header having data identifying the viewing device and a user of the viewing device, a path header having data identifying a path of the user interaction with the augmented reality content, and a sensor header having data identifying a plurality of sensors, the data payload comprising dynamically sized sampling data from the sensor data;

normalizing the path bundle data packages;

aggregating the normalized path bundle data packages; and

performing the analytics computation on the normalized and aggregated path bundle data packages.

2. The server of claim 1 , wherein the display includes a heatmap identifying areas with relatively higher number of users' interactions with the augmented reality content.

3. The server of claim 1 , wherein the display includes trajectory paths of the users' interactions with the augmented reality content, the trajectory paths identifying rotations and movements of the augmented reality content caused by the users' interactions.

4. The server of claim 3 , wherein the display includes an identification of most common trajectory paths of the users' interactions with the augmented reality content.

5. The server of claim 1 , wherein the geometric paths of the users' interactions comprise a geometric trajectory based on a dwelling time exceeding a threshold for a view of the augmented reality content.

6. The server of claim 1 , wherein the operations further comprise generating an extension of the data model to represent the normalized and aggregated path bundle data packages, accessing trigger conditions, to perform performing the analytics computation on the extension of the data model based on the trigger conditions, and identifying a spatial pattern based on the trigger conditions.

7. The server of claim 6 , wherein the operations further comprise normalizing and aggregating spatial and sensor data from multiple sources across multiple dimensions from the path to filter singular and aggregate data.

8. The server of claim 1 , wherein the operations further comprise:

identifying spatial patterns for accidents and event predictions;

recognizing similar patterns and warn operators of the viewing device;

assigning weighting schemes for switching amongst sensors of the viewing device;

identifying user focus points and user loss of focus and redirect the user;

ranking task complexity and calibrate user skill levels based on the normalized and aggregated path bundle data packages; and

identifying spatial movement-based gestures and interfaces based on the normalized and aggregated path bundle data packages.

9. The server of claim 1 , wherein the sensor data is related to a position and an orientation of the viewing device over time, the plurality of sensors comprising optical sensors and inertia sensors,

wherein the contextual header comprises a viewing device identifier, a session identifier, a physical location, and a viewing device type,

wherein the path header comprises a path start time, a path end time, a type of sensor identifier for each corresponding sensor, a total duration of the path, and a total distance of the path,

wherein the sensor header comprises, for each sensor, a sensor identifier, a sensor position identifier, three-dimensional position data, orientation data, and a number of data samples,

wherein the dynamically sized sampling data comprises a sensor identifier, a data sample number, a relative time, a duration percentage through the path, and a time percentage through the path.

10. A method comprising:

performing, using a hardware processor of a machine, an analytics computation of users' interactions with an augmented reality content based on a physical object captured by a viewing device, the analytics computation comprising a computation of geometric paths of the users' interactions with the augmented reality content;

causing a display of a visualization of the analytics computation from the users' interactions with the augmented reality content;

accessing and processing path bundle data packages received from the viewing device, the path bundle data packages describing a user interaction of the viewing device with the augmented reality content based on a physical object captured by the viewing device, the path bundle data packages generated based on sensor data using a data model comprising a data header and a data payload, the data header comprising a contextual header having data identifying the viewing device and a user of the viewing device, a path header having data identifying a path of the user interaction with the augmented reality content, and a sensor header having data identifying a plurality of sensors, the data payload comprising dynamically sized sampling data from the sensor data, the path bundle processing module comprising a normalization module;

normalizing the path bundle data packages;

aggregating the normalized path bundle data packages; and

performing the analytics computation on the normalized and aggregated path bundle data packages.

11. The method of claim 10 , wherein the display includes a heatmap identifying areas with relatively higher number of users' interactions with the augmented reality content.

12. The method of claim 10 , wherein the display includes trajectory paths of the users' interactions with the augmented reality content, the trajectory paths identifying rotations and movements of the augmented reality content caused by the users' interactions.

13. The method of claim 12 , wherein the display includes an identification of most common trajectory paths of the users' interactions with the augmented reality content.

14. The method of claim 10 , wherein the geometric paths of the users' interactions comprise a geometric trajectory based on a dwelling time exceeding a threshold for a view of the augmented reality content.

15. The method of claim 10 , further comprising:

generating an extension of the data model to represent the normalized and aggregated path bundle data packages, to access trigger conditions, to perform the analytics computation on the extension of the data model based on the trigger conditions, and to identify a spatial pattern based on the trigger conditions; and

normalizing and aggregating spatial and sensor data from multiple sources across multiple dimensions from the path to filter singular and aggregate data.

16. The method of claim 10 , further comprising:

identifying spatial patterns for accidents and event predictions;

recognizing similar patterns and warning operators of the viewing device;

assigning weighting schemes for switching amongst sensors of the viewing device;

identifying user focus points and user loss of focus and redirecting the user;

ranking task complexity and calibrating user skill levels based on the normalized and aggregated path bundle data packages; and

identifying spatial movement-based gestures and interfaces based on the normalized and aggregated path bundle data packages.

17. The method of claim 10 , wherein the sensor data is related to a position and an orientation of the viewing device over time, the plurality of sensors comprising optical sensors and inertia sensors, wherein the contextual header comprises a viewing device identifier, a session identifier, a physical location, and a viewing device type, wherein the path header comprises a path start time, a path end time, a type of sensor identifier for each corresponding sensor, a total duration of the path, and a total distance of the path, wherein the sensor header comprises, for each sensor, a sensor identifier, a sensor position identifier, three-dimensional position data, orientation data, and a number of data samples, wherein the dynamically sized sampling data comprises a sensor identifier, a data sample number, a relative time, a duration percentage through the path, and a time percentage through the path.

18. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

performing an analytics computation of users' interactions with an augmented reality content based on a physical object captured by a viewing device, the analytics computation comprising a computation of geometric paths of the users' interactions with the augmented reality content;

causing a display of a visualization of the analytics computation from the users' interactions with the augmented reality content;

accessing and processing path bundle data packages received from the viewing device, the path bundle data packages describing a user interaction of the viewing device with the augmented reality content based on a physical object captured by the viewing device, the path bundle data packages generated based on sensor data using a data model comprising a data header and a data payload, the data header comprising a contextual header having data identifying the viewing device and a user of the viewing device, a path header having data identifying a path of the user interaction with the augmented reality content, and a sensor header having data identifying a plurality of sensors, the data payload comprising dynamically sized sampling data from the sensor data, the path bundle processing module comprising a normalization module;

normalizing the path bundle data packages;

aggregating the normalized path bundle data packages; and

performing the analytics computation on the normalized and aggregated path bundle data packages.

Assignments (12)

Continuity (1)