Method for embedding product information in video using radio frequencey information
View Patent ↗Embodiments are directed towards tracking physical objects' pixel locations in a stream of video frames. Radio frequency (RF) readers may be positioned relative to a scene. RF tags may be positioned in the scene at known pixel locations within the field of view of a video recording device. The RF tags may be enabled to generate RF signals to the RF readers. The RF readers may generate RF values based on the RF signals, which may be employed to determine a function to translate the RF values into known pixel locations within a video frame. An RF tag may be disposed at a physical location of at least one object within the scene. A stream of video frames and RF values may be recorded over time. The function may be employed to translate the recorded RF values into pixel locations for the least one object each recorded video frame.
1. A method for tracking objects in a stream of video frames, comprising:
associating information regarding at least one object with at least one radio frequency (RF) tag, wherein the at least one RF tag is disposed at a physical location of the at least one object;
receiving from the at least one RF tag at least one RF signal at a plurality of RF readers, wherein the RF readers generate a plurality of RF values based on the at least one RF signal;
recording a stream of video frames for a scene that includes the at least one object over time;
recording by each of the plurality of RF readers the plurality of RF values over time for the at least one RF tag;
determining a pixel location within each video frame and a depth value of the at least one object based on the plurality of RF values;
determining a shape of the at least one object using the determined pixel location and the depth value; and
embedding the determined pixel location and the determined shape of the at least one object in one or more video frames of the stream of video frames, comprising:
discarding at least a portion of one or more of the determined pixel location and the determined shape of the at least one object in response to determining that the at least one object is overlapped in the video frame by another object; and
embedding any undiscarded pixel location and shape of the at least one object in the one or more video frames of the stream of video frames.
2. The method of claim 1 , wherein each RF value is based on a received signal strength indicator and transmission power of the at least one RF signal.
3. The method of claim 1 , further comprising:
overlaying a region over the at least one object in the scene based on the determined pixel location in the recorded video frames, wherein an interaction by a user with the region enables at least one action associated with the at least one object.
4. The method of claim 1 , further comprising:
determining a configuration RF value for at least one second RF tag that is positioned within the scene relative to the at least one RF tag disposed at the physical location of the at least one object; and
translating the recorded RF values into a normalized screen position for the at least one object positioned in the scene for each recorded video frame based on the recorded RF values and the configuration RF value for the at least one second RF tag.
5. The method of claim 1 , further comprising:
providing the recorded stream of video frames to at least one display; and
providing information to the display that corresponds to the at least one object based on a mouse click or mouse over of the determined pixel location corresponding with the at least one object positioned in the scene.
6. A network computer that is operative to track objects in a stream of video frames, comprising:
a memory; and
a processor coupled to the memory and configured with processor-executable instructions to perform operations comprising:
associating information regarding at least one object with at least one radio frequency (RF) tag, wherein the at least one RF tag is disposed at a physical location of the at least one object;
receiving from the at least one RF tag at least one RF signal at a plurality of RF readers, wherein the RF readers generate a plurality of RF values based on the at least one RF signal;
obtaining a recorded stream of video frames for a scene that includes the at least one object over time;
obtaining from each of the plurality of RF readers RF values over time for the at least one RF tag;
determining a pixel location within each video frame and a depth value of the at least one object based on the plurality of RF values;
determining a shape of the at least one object using the determined pixel location and the depth value; and
embedding the determined pixel location and the determined shape of the at least one object in one or more video frames of the stream of video frames, comprising:
discarding at least a portion of one or more of the determined pixel location and the determined shape of the at least one object in response to determining that the at least one object is overlapped in the video frame by another object; and
embedding any undiscarded pixel location and shape of the at least one object in the one or more video frames of the stream of video frames.
7. The network computer of claim 6 , wherein each RF value is based on a received signal strength indicator and transmission power of the at least one RF signal.
8. The network computer of claim 6 , wherein the processor is further configured with processor-executable instructions to perform operations comprising:
overlaying a region over the at least one object in the scene based on the determined pixel location in the recorded video frames, wherein an interaction by a user with the region enables at least one action associated with the at least one object.
9. The network computer of claim 6 , wherein the processor is further configured with processor-executable instructions to perform operations comprising:
determining a configuration RF value for at least one second RF tag that is positioned within the scene relative to the at least one RF tag disposed at the physical location of the at least one object; and
translating the recorded RF values into a normalized screen position for the at least one object positioned in the scene for each recorded video frame based on the recorded RF values and the configuration RF value for the at least one second RF tag.
10. The network computer of claim 6 , wherein the processor is further configured with processor-executable instructions to perform operations comprising:
providing the recorded stream of video frames to at least one display; and
providing information to the display that corresponds to the at least one object based on a mouse click or mouse over of the determined pixel location corresponding with the at least one object positioned in the scene.
11. A system for tracking objects in a steam of video frames, comprising:
at least one radio frequency (RF) tag that is disposed at a physical location of at least one object, wherein the at least one RF tag configured to generate at least one RF signal;
a video recording device configured to record a stream of video frames for a scene that includes the at least one object over time;
a plurality of RF readers configured to generate a plurality of RF values based on the at least one RF signal, wherein each of the plurality of RF readers generates RF values over time for the at least one RF tag, wherein the generated RF values are synchronized over time with the recorded stream of video frames; and
a network computer, comprising:
a memory; and
a processor coupled to the memory and configured with processor-executable instructions to perform operations comprising:
determining a pixel location within each video frame and a depth value of the at least one object based on the plurality of RF values;
determining a shape of the at least one object using the determined pixel location and the depth value; and
embedding the determined pixel location and the determined shape of the at least one object in one or more video frames of the stream of video frames, comprising:
discarding at least a portion of one or more of the determined pixel location and the determined shape of the at least one object in response to determining that the at least one object is overlapped in the video frame by another object; and
embedding any undiscarded pixel location and shape of the at least one object in the one or more video frames of the stream of video frames.
12. The system of claim 11 , wherein each RF value is based on a received signal strength and transmission power indicator of the at least one RF signal.
13. The system of claim 11 , wherein the processor is further configured with processor-executable instructions to perform operations comprising:
overlaying a region over the at least one object in the scene based on the determined pixel location in the recorded video frames, wherein an interaction by a user with the region enables at least one action associated with the at least one object.
14. The system of claim 11 , wherein the processor is further configured with processor-executable instructions to perform operations comprising:
determining a configuration RF value for at least one second RF tag that is positioned within the scene relative to the at least one RF tag disposed at the physical location of the at least one object; and
translating the recorded RF values into a normalized screen position for the at least one object positioned in the scene for each recorded video frame based on the recorded RF values and the configuration RF value for the at least one second RF tag.
15. The system of claim 11 , wherein the processor is further configured with processor-executable instructions to perform operations comprising:
providing the recorded stream of video frames to at least one display; and
providing information to the display that corresponds to the at least one object based on a mouse click or mouse over of the determined pixel location corresponding with the at least one object positioned in the scene.