Granted Patent B2

US 9,607,584 · App. 13/840,359 · Granted Mar 28, 2017

Inventor: Brian Mullins (Garden Grove, CA)

Assignee: DAQRI, LLC

G09G5/377G09G3/001G09G3/003G09G2370/022

Abstract

Claims (54)

1. A computer-implemented method comprising:

receiving, from a plurality of devices, analytics data describing user interactions with a physical object, the analytics data including pose data indicating locations on the physical object where optical sensors of the plurality of devices were directed while users interacted with the physical object as well as time durations that the optical sensors were directed at the locations on the physical object;

determining, based on the analytics data, a frequency that users of the plurality of devices looked at the locations of the physical object;

generating, by a computer processor, a visualization content dataset for the physical object, the visualization content data set comprising a set of images of the physical object and corresponding analytics virtual object models to be engaged with each image of the physical object, the analytics virtual object model for each image indicating a frequency that users of the plurality of devices looked at the locations of the physical object captured in the image; and

transmitting the visualization content dataset for the physical object to at least a first device, wherein the first device uses the visualization content dataset to render a heat-map over a live image of the physical object captured by an optical sensor of the first device, the heat map indicating the frequency that users of the plurality of devices looked at the locations of the physical object captured in the live image.

2. The computer-implemented method of claim 1 , further comprising:

generating, for each image of the physical object, the analytics virtual object model indicating the frequency that users of the plurality of device looked at the locations of the physical object captured in the image.

3. The computer-implemented method of claim 1 , further comprising:

determining a pose estimation of a device relative to the physical object, a pose duration of the device relative to the physical object, a pose orientation of the device relative to the physical object, and a pose interaction of the device relative to the physical object.

4. The computer-implemented method of claim 3 , wherein the pose estimation comprises a location on the physical object aimed at by the device;

wherein the pose duration comprises a time duration within which the device is aimed at a same location on the physical object;

wherein the pose orientation comprises an orientation of the device aimed at the physical object; and

wherein the pose interaction comprises interactions of the user on the device with respect to the physical object.

5. The computer-implemented method of claim 4 , further comprising:

generating the visualization content dataset for multiple devices based on the pose estimation, the pose duration, the pose orientation, and the pose interaction from multiple devices.

6. The computer-implemented method of claim 4 , further comprising:

generating the visualization content dataset for the device based on the pose estimation, the pose duration, the pose orientation, and the pose interaction from the device.

7. The computer-implemented method of claim 1 , further comprising:

storing a primary content dataset and a contextual content dataset, the primary content dataset comprising a first set of images and corresponding analytics virtual object models, the contextual content dataset comprising a second set of images and corresponding analytics virtual object models.

8. The computer-implemented method of claim 7 , further comprising:

determining that a captured image received from a device is not recognized in the primary content dataset; and

generating the contextual content dataset for the device.

9. The computer-implemented method of claim 1 , wherein the analytics data comprises usage conditions of a device, the usage conditions of the device comprising social information of a user of the device, location usage information, and time information of the device.

10. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a machine, cause the machine to:

receive, from a plurality of devices, analytics data describing user interactions with a physical object, the analytics data including pose data indicating locations on the physical object where optical sensors of the plurality of devices were directed while users interacted with the physical object as well as time durations that the optical sensors were directed at the locations on the physical object;

determine, based on the analytics data, a frequency that users of the plurality of devices looked at the locations of the physical object;

generate a visualization content dataset for the physical object, the visualization content data set comprising a set of images of the physical object and corresponding analytics virtual object models to be engaged with each image of the physical object, the analytics virtual object model for each image indicating a frequency that users of the plurality of devices looked at the locations of the physical object captured in the image; and

transmitting the visualization content dataset for the physical object to at least a first device, wherein the first device uses the visualization content dataset to render a heat-map over a live image of the physical object captured by an optical sensor of the first device, the heat map indicating the frequency that users of the plurality of devices looked at the locations of the physical object captured in the live image.

11. A server comprising:

one or more computer processors; and

one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the server to:

receive, from a plurality of devices, analytics data describing user interactions with a physical object, the analytics data including pose data indicating locations on the physical object where optical sensors of the plurality of devices were directed while users interacted with the physical object as well as time durations that the optical sensors were directed at the locations on the physical object;

determine, based on the analytics data, a frequency that users of the plurality of devices looked at the locations of the physical object;

generate a visualization content dataset for the physical object, the visualization content data set comprising a set of images of the physical object and corresponding analytics virtual object models to be engaged with each image of the physical object, the analytics virtual object model for each image indicating a frequency that users of the plurality of devices looked at the locations of the physical object captured in the image; and

transmit the visualization content dataset for the physical object to at least a first device, wherein the first device uses the visualization content dataset to render a heat-map over a live image of the physical object captured by an optical sensor of the first device, the heat map indicating the frequency that users of the plurality of devices looked at the locations of the physical object captured in the live image.

12. The server of claim 11 , wherein the instructions further cause the server to:

generate, for each image of the physical object, the analytics virtual object model indicating the frequency that users of the plurality of device looked at the locations of the physical object captured in the image.

13. The server of claim 11 , wherein the instructions further cause the server to:

determine a pose estimation of a device relative to the physical object, a pose duration of the device relative to the physical object, a pose orientation of the device relative to the physical object, and a pose interaction of the device relative to the physical object.

14. The server of claim 13 , wherein the pose estimation comprises a location on the physical object aimed at by the device;

wherein the pose duration comprises a time duration within which the device is aimed at a same location on the physical object;

wherein the pose orientation comprises an orientation of the device aimed at the physical object; and

wherein the pose interaction comprises interactions of the user on the device with respect to the physical object.

15. The server of claim 14 , wherein the instructions further cause the server to:

generate the visualization content dataset for multiple devices based on the pose estimation, the pose duration, the pose orientation, and the pose interaction from multiple devices.

16. The server of claim 14 , wherein the instructions further cause the server to:

generate the visualization content dataset for the device based on the pose estimation, the pose duration, the pose orientation, and the pose interaction from the device.

17. The server of claim 11 , wherein the instructions further cause the server to:

store a primary content dataset and a contextual content dataset, the primary content dataset comprising a first set of images and corresponding analytics virtual object models, the contextual content dataset comprising a second set of images and corresponding analytics virtual object models.

18. The server of claim 17 , wherein the instructions further cause server to:

determine that a captured image received from a device is not recognized in the primary content dataset; and

generate the contextual content dataset for the device.

19. The server of claim 11 , wherein the analytics data comprises usage conditions of a device.

20. The server of claim 19 , wherein the usage conditions of the device comprises social information of a user of the device, location usage information, and time information of the device.

Assignments (14)

Continuity (1)