Tracked video zooming
Systems, methods, and instrumentalities are disclosed for dynamic picture-in-picture (PIP) by a client. The client may reside on any device. The client may receive video content from a server, and identify an object within the video content using at least one of object recognition or metadata. The metadata may include information that indicates a location of an object within a frame of the video content. The client may receive a selection of the object by a user, and determine positional data of the object across frames of the video content using at least one of object recognition or metadata. The client may display an enlarged and time-delayed version of the object within a PIP window across the frames of the video content. Alternatively or additionally, the location of the PIP window within each frame may be fixed or may be based on the location of the object within each frame.
1. A method for generating a dynamic picture-in-picture for displaying on a display device, the method comprising:
receiving video content from a server;
determining a first position of a tracked object among a plurality of objects within a first frame of the video content based on object recognition or metadata;
determining a position of a first window based on the first position of the tracked object, wherein the position of the first window is overlapping with the tracked object, wherein the first window comprises a first portion of the first frame, wherein the first portion of the first frame is visually enlarged, and wherein the tracked object, within the first portion of the first frame, is visually enlarged;
generating the first window within the first frame for displaying on the display device;
determining a second position of the tracked object among the plurality of objects within a second frame of the video content based on object recognition or metadata, wherein the second frame is temporally subsequent to the first frame, and wherein the second position of the tracked object is different than the first position of the tracked object;
determining a position of a second window based on the second position of the tracked object, wherein the position of the second window is overlapping with the tracked object, wherein the second window comprises a second portion of the second frame, wherein the second portion of the second frame is visually enlarged, and wherein the tracked object, within the second portion of the second frame, is visually enlarged; and
generating the second window within the second frame for displaying on the display device.
2. The method of claim 1 , further comprising determining that the first window or the second window is overlapping with the tracked object.
3. The method of claim 1 , further comprising:
determining a third position of the tracked object among the plurality of objects within a third frame of the video content based on object recognition or metadata; and
generating a third window in a predetermined location within a fourth frame for displaying on the display device, wherein a third portion of the third frame is within the third window, wherein the third portion of the third frame is visually enlarged, wherein the tracked object, within the third portion of the third frame is visually enlarged, and wherein the fourth frame is temporally subsequent to the third frame.
4. The method of claim 1 , wherein the first window comprises the first portion of the first frame based on a user selection of the tracked object.
5. The method of claim 1 , further comprising:
identifying the plurality of objects within an earlier frame of the video content, the plurality of objects comprising the tracked object;
generating a plurality of windows within the earlier frame for displaying on the display device, each of the plurality of windows comprising a respective tracked object among the plurality of objects, wherein each of the plurality of windows provides an indication of the respective tracked object; and
cycling through a window of focus of the plurality of windows based on user input.
6. The method of claim 5 , further comprising:
receiving a user selection of the tracked object among the plurality of objects; and
enlarging the tracked object within the first window based on the user selection.
7. The method of claim 1 , wherein metadata comprises information indicating a location of the tracked object within a frame of the video content.
8. The method of claim 1 , further comprising generating information for displaying on the display device relating to the tracked object within the first frame or the second frame.
9. A device comprising:
a processor configured to at least:
receive video content from a server;
determine a first position of a tracked object among a plurality of objects within a first frame of the video content based on object recognition or metadata;
determine a position of a first window based on the first position of the tracked object, wherein the position of the first window is overlapping with the tracked object, wherein the first window comprises a first portion of the first frame, wherein the first portion of the first frame is visually enlarged, and wherein the tracked object, within the first portion of the first frame, is visually enlarged;
generate the first window within the first frame for displaying on a display device;
determine a second position of the tracked object among the plurality of objects within a second frame of the video content based on object recognition or metadata, wherein the second frame is temporally subsequent to the first frame, and wherein the second position of the tracked object is different than the first position of the tracked object;
determine a position of a second window based on the second position of the tracked object, wherein the position of the second window is overlapping with the tracked object, wherein the second window comprises a second portion of the second frame, wherein the second portion of the second frame is visually enlarged, and wherein the tracked object, within the second portion of the second frame, is visually enlarged; and
generate the second window within the second frame for displaying on the display device.
10. The device of claim 9 , wherein the processor is configured to determine that the first window or the second window is overlapping with the tracked object.
11. The device of claim 9 , wherein the processor is configured to:
determine a third position of the tracked object among the plurality of objects within a third frame of the video content based on object recognition or metadata; and
generate a third window in a predetermined location within a fourth frame for displaying on the display device, wherein a third portion of the third frame is within the third window, wherein the third portion of the third frame is visually enlarged, wherein the tracked object, within the third portion of the third frame is visually enlarged, and wherein the fourth frame is temporally subsequent to the third frame.
12. The device of claim 9 , wherein the first window comprises the first portion of the first frame based on a user selection of the tracked object.
13. The device of claim 9 , wherein the processor is configured to:
identify the plurality of objects within an earlier frame of the video content, the plurality of objects comprising the tracked object;
generate a plurality of windows within the earlier frame for displaying on the display device, each of the plurality of windows comprising a respective tracked object among the plurality of objects, wherein each of the plurality of windows provides an indication of the respective tracked object; and
cycle through a window of focus of the plurality of windows based on user input.
14. The device of claim 13 , wherein the processor is configured to:
receive a user selection of the tracked object among the plurality of objects; and
enlarge the tracked object within the first window based on the user selection.
15. The device of claim 9 , wherein metadata comprises information indicating a location of the tracked object within a frame of the video content.
16. The device of claim 9 , wherein the processor is configured to generate information for displaying on the display device relating to the tracked object within the first frame or the second frame.