Granted Patent B2

US 11,032,603 · App. 16/222,275 · Granted Jun 8, 2021

Inventor: Brian Mullins (Altadena, CA)

Assignee: RPX Corporation

H04N21/4302G06T19/006H04N21/242H04N21/4223H04N21/6106H04N21/816

Abstract

Claims (57)

1. A server comprising:

one or more hardware processor comprising a recording application, the recording application being configured to perform operations comprising:

receiving first content data from a first device, the first content data having been generated by the first device and being associated with performance of a first task in relation to a first physical device;

providing the first content data to a second device;

receiving second content data from the second device, the second content data including modifications to the first content data, the modification including an annotation describing performance of the first task in relation to the first physical device;

forming playback parameters based on a context of the first device, the context of the first device being based on an orientation of the first device in relation to the first physical device and the first task;

receiving third content data from a third device, the third content data describing a context of the third device, the context of the third device indicating an orientation of the third device in relation to the first physical device and identifying the first task;

determining, based on the context of the third device, that the third device meets the playback parameters;

in response to determining that the third device meets the playback parameters, generating an enhanced playback session based on the first content data and second content data, the enhanced playback session including the first content data generated by the first device modified based on the modifications included in the second content data; and

communicating the enhanced playback session to the third device.

2. The server of claim 1 , wherein the first content data includes first sensor data and a request for assistance identifying the context of the first device, and

wherein the operations further comprise: identifying the second device based on the context of the first device in response to the request for assistance.

3. The server of claim 2 , wherein the second content data includes second sensor data of the second device and the modifications to the first content data, the operations further comprising:

synchronizing the first content data with the second content data based on the first sensor data and second sensor data.

4. The server of claim 1 , wherein the context of the first device is based on sensor data included in the first content data, a task of a first user of the first device, an identity of the first user, and a time parameter.

5. The server of claim 1 , wherein the first content data of the first device comprises at least one of video data, image data, audio data, graphic data, and three-dimensional model data, and wherein the first content data further includes first sensor data of the first device comprising at least one of geographic data, inertia data, and orientation data.

6. The server of claim 1 , wherein the second content data of the second device comprises at least one of video data, image data, audio data, graphic data, three-dimensional model data, and

wherein the second content data of the second device comprises at least one of annotation data related to the first content data, and manipulation data related to a second user of the second device manipulating a three-dimensional model of a physical object identified in the first content data.

7. The server of claim 1 , wherein the playback parameters comprise at least one of a location parameter, a user parameter, and a task parameter.

8. The server of claim 1 , wherein the enhanced playback session comprises a video recording synchronized to a user task in the context of the first device, or an augmented reality content synchronized to a user task in the context of the first device.

9. The server of claim 1 , wherein the operations further comprise:

identifying a manipulation of virtual objects displayed in the second device, the virtual objects rendered based on a physical object viewed by the first device, the virtual objects displayed in the first device in relation to a reference object viewed with the first device; and

displaying the manipulation of the virtual objects at the first device.

10. The server of claim 1 , wherein the operations further comprise:

contacting the second device in response to determining that the third device meets the playback parameters; and

enabling the third device to communicate with the second device in response to contacting the second device.

11. A method comprising:

receiving first content data from a first device, the first content data having been generated by the first device and being associated with performance of a first task in relation to a first physical device;

providing the first content data to a second device;

receiving second content data from the second device, the second content data including modifications to the first content data, the modification including an annotation describing performance of the first task in relation to the first physical device;

forming playback parameters based on a context of the first device, the context of the first device being based on an orientation of the first device in relation to the first physical device and the first task;

receiving third content data from a third device, the third content data describing a context of the third device, the context of the third device indicating an orientation of the third device in relation to the first physical device and identifying the first task;

determining, based on the context of the third device, that the third device meets the playback parameters;

in response to determining that the third device meets the playback parameters, generating an enhanced playback session based on the first content data and second content data, the enhanced playback session including the first content data generated by the first device modified based on the modifications included in the second content data; and

communicating the enhanced playback session to the third device.

12. The method of claim 11 , wherein the first content data includes first sensor data and a request for assistance identifying the context of the first device, and

wherein the method further comprises: identifying the second device based on the context of the first device in response to the request for assistance.

13. The method of claim 12 , wherein the second content data includes second sensor data of the second device and modifications to the first content data, the method further comprising:

synchronizing the first content data with the second content data based on the first and second sensor data.

14. The method of claim 11 , wherein the context of the first device is based on sensor data included in the first content data, a task of a first user of the first device, an identity of the first user, and a time parameter.

15. The method of claim 11 , wherein the first content data of the first device comprises at least one of video data, image data, audio data, graphic data, and three-dimensional model data, and wherein the first content data further includes first sensor data of the first device comprising at least one of geographic data, inertia data, and orientation data.

16. The method of claim 11 , wherein the second content data of the second device comprises at least one of video data, image data, audio data, graphic data, three-dimensional model data, and

wherein the second content data of the second device comprises at least one of annotation data related to the first content data, and manipulation data related to a second user of the second device manipulating a three-dimensional model of a physical object identified in the first content data.

17. The method of claim 11 , wherein the playback parameters comprise at least one of a location parameter, a user parameter, and a task parameter.

18. The method of claim 11 , wherein the enhanced playback session comprises a video recording synchronized to a user task in the context of the first device, or an augmented reality content synchronized to a user task in the context of the first device.

19. The method of claim 11 , further comprising:

identifying a manipulation of virtual objects displayed in the second device, the virtual objects rendered based on a physical object viewed by the first device, the virtual objects displayed in the first device in relation to a reference object viewed with the first device; and

displaying the manipulation of the virtual objects at the first device.

20. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a computer, cause the computer to perform operations comprising:

receiving first content data from a first device, the first content data having been generated by the first device and being associated with performance of a first task in relation to a first physical device;

providing the first content data to a second device;

receiving second content data from the second device, the second content data including modifications to the first content data, the modification including an annotation describing performance of the first task in relation to the first physical device;

forming playback parameters based on a context of the first device, the context of the first device being based on an orientation of the first device in relation to the first physical device and the first task;

receiving third content data from a third device, the third content data describing a context of the third device, the context of the third device indicating an orientation of the third device in relation to the first physical device and identifying the first task;

determining, based on the context of the third device, that the third device meets the playback parameters;

in response to determining that the third device meets the playback parameters, generating an enhanced playback session based on the first content data and second content data, the enhanced playback session including the first content data generated by the first device modified based on the modifications included in the second content data; and

communicating the enhanced playback session to the third device.

Assignments (12)

Continuity (3)

Cited By (1)

US 12,456,266