Method and electronic device for object tracking in a light-field capture
View Patent ↗A method and an electronic device for object tracking in a sequence of light-field captures. A data acquisition unit acquires a sequence of light-field captures, wherein each light-field capture comprises a plurality of views. A feature determining unit determines features of an initial visual appearance model for an object of interest in a reference view of a first light-field capture. A feature matching unit matches the features in the reference view and in the further views of the first light-field capture. A feature discarding unit discards features that cannot be well matched in all views of the first light-field capture. An appearance model building unit builds an updated visual appearance model for the object of interest based on the remaining features. Finally, a movement tracking unit tracks the movement of the object of interest in the sequence of light-field captures using the visual appearance model.
1. A method for object tracking in a sequence of light-field captures, the method comprising:
acquiring a sequence of light-field captures with a light-field camera, each light-field capture having a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, one view among the plurality of views being a reference view;
determining features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
matching the features in the reference view and in the further views of the first light-field capture;
discarding features that cannot be matched in all or most of the views of the first light-field capture;
building an updated visual appearance model for the object of interest based on the remaining features; and
tracking a movement of the object of interest in the sequence of light-field captures using the visual appearance model.
2. The method according to claim 1 , wherein the views of each light-field capture are acquired in a timely synchronized manner.
3. The method according to claim 1 , wherein the matching of the features in the reference view and in the further views of the first light-field capture is based on a similarity metric score.
4. A non-transitory computer readable storage medium having stored therein instructions enabling object tracking in a sequence of light-field captures, which, when executed by a computer, cause the computer to:
acquire a sequence of light-field captures with a light-field camera, each light-field capture having a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, and one view among the plurality of views being a reference view;
determine features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
match the features in the reference view and in the further views of the first light-field capture;
discard features that cannot be matched in all or most of the views of the first light-field capture;
build an updated visual appearance model for the object of interest based on the remaining features; and
track a movement of the object of interest in the sequence of light-field captures using the visual appearance model.
5. The non-transitory computer readable storage medium according to claim 4 , wherein the instructions cause the computer to acquire the views of each light-field capture in a timely synchronized manner.
6. The non-transitory computer readable storage medium according to claim 4 , wherein the matching of the features in the reference view and in the further views of the first light-field capture is based on a similarity metric score.
7. An electronic device for object tracking in a sequence of light-field captures, wherein the electronic device comprises:
a processor coupled to a memory, the processor being configured to:
acquire a sequence of light-field captures with a light-field camera, wherein each light-field capture comprises a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, one view among the plurality of views being a reference view;
determine features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
match the features in the reference view and in the further views of the first light-field capture;
discard features that cannot be matched in all or most of the views of the first light-field capture;
build an updated visual appearance model for the object of interest based on the remaining features;
track a movement of the object of interest in the sequence of light-field captures using the visual appearance model; and
output data relative to the movement of the object of interest in the sequence of light-field captures.
8. The electronic device according to claim 7 , wherein the processor is coupled to a light-field camera.
9. The electronic device according to claim 7 , wherein the processor is configured to acquire the views of each light-field capture in a timely synchronized manner.
10. An electronic device for object tracking in a sequence of light-field captures, the electronic device comprising a processing device and a memory device having stored therein instructions, which, when executed by the processing device, cause the electronic device to:
acquire a sequence of light-field captures with a light-field camera, each light-field capture having a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, one view among the plurality of views being a reference view;
determine features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
match the features in the reference view and in the further views of the first light-field capture;
discard features that cannot be matched in all or most of the views of the first light-field capture;
build an updated visual appearance model for the object of interest based on the remaining features; and
track a movement of the object of interest in the sequence of light-field captures using the visual appearance model.
11. The electronic device according to claim 7 , wherein the processor is configured to match the features in the reference view and in the further views of the first light-field capture based on a similarity metric score.
12. An apparatus for processing image data, in particular video post-production system or video surveillance system, comprising the electronic device according to claim 7 or 10 .
13. The electronic device according to claim 10 , wherein the electronic device is coupled to a light-field camera.
14. The electronic device according to claim 10 , wherein the instructions cause the electronic device to acquire the views of each light-field capture in a timely synchronized manner.
15. The electronic device according to claim 10 , wherein the processing device causes the electronic device to match the features in the reference view and in the further views of the first light-field capture based on a similarity metric score.