Techniques for aligning visual content to three-dimensional signals
A system and method for visual content alignment. A method includes identifying subjects in visual content, wherein the visual content is captured by a camera deployed at a site, wherein a position sensor is deployed with a target subject among the subjects, wherein the position sensor captures signals representing positions of the target subject at the site; transforming points representing the positions of the target subject from a first feature space to a second feature space, wherein the first feature space is a feature space of the signals, wherein the second feature space is a feature space of the visual content; and mapping the positions of the target subject represented by the signals to positions within the visual content based on the points in the second feature space.
1 . A method for visual content alignment, comprising:
identifying a plurality of subjects in visual content, wherein the visual content is captured by a camera deployed at a site, wherein a position sensor is deployed with a target subject among the plurality of subjects, wherein the position sensor captures a plurality of signals representing a plurality of positions of the target subject at the site;
transforming a plurality of points representing the plurality of positions of the target subject from a first feature space to a second feature space, wherein the first feature space is a feature space of the plurality of signals, wherein the second feature space is a feature space of the visual content, wherein transforming the plurality of points includes using an affine transformation, wherein the plurality of points representing the plurality of positions of the target subject in the first feature space form a curve, wherein transforming the plurality of points further includes classifying a plurality of portions of the curve into a plurality of classifications and mapping the plurality of portions of the curve to the plane of the second feature space based on a parameterization of the curve, wherein the plurality of points is transformed based on the plurality of classifications and the mapping; and
mapping the plurality of positions of the target subject represented by the plurality of signals to a plurality of positions within the visual content based on the plurality of points in the second feature space.
2 . The method of claim 1 , wherein the plurality of points is transformed using a transformation which preserves lines and parallelism.
3 . The method of claim 1 , wherein transforming the plurality of points further comprises:
solving for an affine value, wherein the plurality of points is transformed based on the affine value.
4 . The method of claim 1 , wherein the visual content includes a plurality of visual content items, wherein mapping the plurality of positions of the target subject represented by the plurality of signals to the plurality of positions within the visual content further comprises:
determining a set of visual content coordinates for each of the plurality of positions, wherein the set of visual content coordinates of each position is defined with respect to a portion of a visual content item among the plurality of visual content items.
5 . The method of claim 1 , further comprising:
identifying an interaction between the target subject and another subject of the plurality of subjects shown in the visual content, wherein the interaction is identified with respect to a position among the plurality of positions represented by the plurality of signals.
6 . The method of claim 5 , wherein the interaction is identified based on a distance between the target subject and the other subject below a threshold.
7 . The method of claim 5 , wherein the visual content includes a plurality of visual content items, further comprising:
identifying the target subject in at least one visual content item of the plurality of visual content items, wherein the interaction is identified based on the at least one visual content item.
8 . A non-transitory computer-readable medium having stored thereon instructions for causing a processing circuitry to execute a process, the process comprising:
identifying a plurality of subjects in visual content, wherein the visual content is captured by a camera deployed at a site, wherein a position sensor is deployed with a target subject among the plurality of subjects, wherein the position sensor captures a plurality of signals representing a plurality of positions of the target subject at the site;
transforming a plurality of points representing the plurality of positions of the target subject from a first feature space to a second feature space, wherein the first feature space is a feature space of the plurality of signals, wherein the second feature space is a feature space of the visual content, wherein transforming the plurality of points includes using an affine transformation, wherein the plurality of points representing the plurality of positions of the target subject in the first feature space form a curve, wherein transforming the plurality of points further includes classifying a plurality of portions of the curve into a plurality of classifications and mapping the plurality of portions of the curve to the plane of the second feature space based on a parameterization of the curve, wherein the plurality of points is transformed based on the plurality of classifications and the mapping; and
mapping the plurality of positions of the target subject represented by the plurality of signals to a plurality of positions within the visual content based on the plurality of points in the second feature space.
9 . A system for visual content alignment, comprising:
a processing circuitry; and
a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:
identify a plurality of subjects in visual content, wherein the visual content is captured by a camera deployed at a site, wherein a position sensor is deployed with a target subject among the plurality of subjects, wherein the position sensor captures a plurality of signals representing a plurality of positions of the target subject at the site;
transform a plurality of points representing the plurality of positions of the target subject from a first feature space to a second feature space, wherein the first feature space is a feature space of the plurality of signals, wherein the second feature space is a feature space of the visual content, wherein transforming the plurality of points includes using an affine transformation, wherein the plurality of points representing the plurality of positions of the target subject in the first feature space form a curve, wherein transforming the plurality of points further includes classifying a plurality of portions of the curve into a plurality of classifications and mapping the plurality of portions of the curve to the plane of the second feature space based on a parameterization of the curve, wherein the plurality of points is transformed based on the plurality of classifications and the mapping; and
map the plurality of positions of the target subject represented by the plurality of signals to a plurality of positions within the visual content based on the plurality of points in the second feature space.
10 . The system of claim 9 , wherein the plurality of points is transformed using a transformation which preserves lines and parallelism.
11 . The system of claim 9 , wherein the system is further configured to:
solve for an affine value, wherein the plurality of points is transformed based on the affine value.
12 . The system of claim 9 , wherein the visual content includes a plurality of visual content items, wherein the system is further configured to:
determine a set of visual content coordinates for each of the plurality of positions, wherein the set of visual content coordinates of each position is defined with respect to a portion of a visual content item among the plurality of visual content items.
13 . The system of claim 9 , wherein the system is further configured to:
identify an interaction between the target subject and another subject of the plurality of subjects shown in the visual content, wherein the interaction is identified with respect to a position among the plurality of positions represented by the plurality of signals.
14 . The system of claim 13 , wherein the interaction is identified based on a distance between the target subject and the other subject below a threshold.
15 . The system of claim 13 , wherein the visual content includes a plurality of visual content items, wherein the system is further configured to:
identify the target subject in at least one visual content item of the plurality of visual content items, wherein the interaction is identified based on the at least one visual content item.