Remote annotation of live video feed
The invention is directed towards a system where a head-mounted display (HMD), or other head mounted computing device, may annotate video data captured by the HMD. When a video is annotated on the computer processor, the data packet associated with that digital marker is encoded as an operation-encoded audio packet and sent over a secure audio link to the HMD. Sending the operation-encoded audio packet over the secure audio link requires the HMD to decode the packet into a data packet which may then be used to annotate a display on the HMD. The user of the HMD may be able to view his own recorded field of view on a display device which may then be annotated using the data in the data packet.
1. A head-mounted computing device comprising:
a visual display module;
one or more processors;
one or more memory devices storing programmable instructions thereon that, when executed by the one or more processors, cause the one or more processors to execute operations comprising:
streaming video data captured via a camera to a remote user device, wherein the video data is displayed via the visual display module as it is being captured;
receiving an audio stream from the remote user device while the video data is being streamed;
detecting an audio packet that is included in the audio stream, the audio packet defining an annotation that corresponds to an object in the video data;
decoding the audio packet to extract the annotation for the object; and
modifying the streamed video data to include the annotation corresponding to the object.
2. The device of claim 1 , wherein a user device encodes the annotation that corresponds to t he object in the video data.
3. The device of claim 2 , wherein the user device data packet includes instructions for a computer processor to modify an image on an application installed on the head-mounted computing device.
4. The device of claim 3 , wherein the user device data packet includes a first location on the image, the first location being a set of coordinates associated with the image indicating where a user indicates to modify the image.
5. The device of claim 4 , wherein the set of coordinates are a three-dimensional set of coordinates.
6. The device of claim 5 , wherein the first location includes the three-dimensional set of coordinates based and a set of location tracking information.
7. The device of claim 6 , wherein the set of location tracking information includes a set of inertial measurements detected by the head-mounted computing device related to a position of the object in a field of view of the head-mounted computing device in relation to the position of the head-mounted computing device.
8. The device of claim 1 , wherein the visual display module uses a communication network application to cause the display of an image.
9. The device of claim 8 , wherein the modifying the display of the image uses a modification application which causes to be displayed as an overlay over the image, wherein the overlay includes the modification.
10. A non-transitory computer storage medium storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
streaming video data captured via a camera on a head-mounted computing device to a remote user device, wherein the video data is displayed via a visual display module as it is being captured;
receiving, an audio stream from the remote user device while the video data is being streamed;
detecting an audio packet that is included in the audio stream, the audio packet defining an annotation that corresponds to an object in the video data;
decoding the audio packet to extract the annotation for the object; and
modifying the streamed video data to include the annotation corresponding to the object.
11. The computer storage medium of claim 10 , wherein the secure audio link is established between the one or more computing devices and the head-mounted computing device using a communication application.
12. The computer storage medium of claim 11 , wherein the annotation for the object includes instructions for a second application installed on the head-mounted computing device to modify an image.
13. The computer storage medium of claim 11 , wherein the annotation for the object includes a first location on an image, the first location being a set of three-dimensional coordinates associated with the captured indicating where a user desires to modify the image and a set of location tracking information.
14. The computer storage medium of claim 13 , wherein the set of location tracking information includes a set of inertial measurements detected by the head-mounted computing device related to a position of the object in a field of view of the head-mounted computing device in relation to the position of the head-mounted computing device.
15. The computer storage medium of claim 14 , further comprising causing a display of the image on the head-mounted computing device to be modified.
16. The computer storage medium of claim 15 , wherein the display of the image on the head-mounted computing device is modified such that the object and the modification are spatially linked based on a three-dimensional location of the object and a three-dimensional location of the modification.
17. A computer-implemented method comprising:
streaming video data captured via a camera on a head-mounted computing device to a remote user device, wherein the video data is displayed via a visual display module as it is being capture;
receiving, an audio stream from the remote user device while the video data is being streamed;
detecting an audio packet that is included in the audio stream, the audio packet defining an annotation that corresponds to an object in the video data;
decoding the audio packet to extract the annotation for the object; and
modifying the streamed video data to include the annotation corresponding to the object.
18. The method of claim 17 , wherein the the annotation for the object includes instructions for a computer processor to modify an image on a second application installed on the head-mounted computing device, a first location of the object associated with the modification within the image, the first location being a set of three-dimensional coordinates and a set of location tracking information.
19. The method of claim 18 , wherein the set of location tracking information includes a set of inertial measurements detected by the head-mounted computing device related to a position of the object in a field of view of the head-mounted computing device in relation to the position of the head-mounted computing device.
20. The method of claim 17 , wherein the visual display module uses a communication network application to cause the display of an image on the head-mounted computing device and uses a modification application which causes to be displayed an overlay over the image, wherein the overlay includes the modification.