Method for processing a video scene and corresponding device
View Patent ↗The present disclosure relates to a new method to make a 3D acquisition of the video scene and to replace an object of the video scene by a substitute object corresponding to the product having substantially common 3D features than the object to be replaced and acceptable criteria values. It permits to minimize the appearance of hidden surfaces or occlusions.
1. A device comprising:
a network interface configured for receiving a 3D representation of a scene, and
a processing unit for determining into said 3D representation at least two candidate objects, for computing a value of at least one criteria for each of said at least two candidate objects, for computing a signature of each of said at least two candidate objects, said signature being representative of 3D features of each of said at least two candidate objects, for automatically determining an object to be removed and a substitute object from a plurality of candidate substitute objects of a library, each candidate substitute object having a signature, said automatic determination being based on comparison of signatures and on the candidate object criteria values, and for processing a video of the scene by replacing the object to be removed by said substitute object into said video of the scene.
2. The device according to claim 1 , wherein the values of at least one criteria for the at least two candidate objects are determined by a user.
3. The device according to claim 1 wherein the at least one criteria is chosen from:
a geographical position of said object in the video of the scene;
a localisation of said object into the foreground of the video of the scene;
a visibility duration of said object in the video of the scene.
4. The device according to claim 1 , wherein the values of at least one criteria for the at least two candidate objects are determined by a user.
5. The device according to claim 1 wherein the at least one criteria is chosen from:
a geographical position of said object in the video of the scene;
a localisation of said object into the foreground of the video of the scene;
a visibility duration of said object in the video of the scene.
6. The device according to claim 1 , wherein said at least one candidate object comprises a cloud of 3D points.
7. The device according to claim 1 , wherein in the library comprising a plurality of candidate substitute objects classified into object families, the substitute object is determined among candidate substitute objects belonging to at least one selected object families.
8. The device according to claim 7 , wherein the selected object families are chosen by a user.
9. The device according to claim 1 , wherein the signature of each object comprises 3D features representative of the geometry of the object, called geometrical features, or 3D features representative of the viewpoint of the object, called viewpoint features.
10. The device according to claim 9 , wherein the signature of an object is at least one histogram among the group of following histograms:
A Viewpoint Feature Histogram;
A Clustered Viewpoint Feature Histogram;
A Point Feature Histogram;
A Fast Point Feature Histogram.
11. The device according to claim 10 , wherein the processing unit is further configured to compute the signatures of the candidate substitute objects of the library.
12. The device according to claim 1 , wherein the substitute object is determined by:
comparing the signatures of the at least two candidate objects with the signatures of the candidate substitute objects of the library;
comparing the candidate object criteria values of the at least two candidate objects, and
identifying the candidate substitute object of the library having a signature identical to or close to the signature of one of the at least two candidate objects, the criteria value of said one of the at least two candidate objects being higher than the criteria value of any other of the at least two candidate objects, the substitute object being the identified candidate substitute object of the library.
13. The device according to claim 1 , wherein the signatures of the candidate substitute objects of the library are pre-computed and stored in the library.
14. A device comprising a network interface configured for receiving a 3D representation of a scene, and at least one processor configured to:
determine into said 3D representation at least two candidate objects,
compute a value of at least one criteria for each of said at least two candidate objects,
compute a signature of each of said at least two candidate objects, said signature being representative of 3D features of each of said at least two candidate objects,
automatically determine an object to be removed and a substitute object from a plurality of candidate substitute objects of a library, each candidate substitute object having a signature, said automatic determination being based on comparison of signatures and on the candidate object criteria values, and
process a video of the scene by replacing the object to be removed by said substitute object into said video of the scene.
15. The device according to claim 14 , wherein said at least one candidate object comprises a cloud of 3D points.
16. The device according to claim 14 , wherein in the library comprising a plurality of candidate substitute objects classified into object families, the substitute object is determined among candidate substitute objects belonging to at least one selected object families.
17. The device according to claim 16 , wherein the selected object families are chosen by a user.
18. The device according to claim 14 , wherein the signature of each object comprises 3D features representative of the geometry of the object, called geometrical features, or 3D features representative of the viewpoint of the object, called viewpoint features.
19. The device according to claim 18 , wherein the signature of an object is at least one histogram among the group of following histograms:
A Viewpoint Feature Histogram;
A Clustered Viewpoint Feature Histogram;
A Point Feature Histogram;
A Fast Point Feature Histogram.
20. The device according to claim 19 , wherein the processor is further configured to compute the signatures of the candidate substitute objects of the library.
21. The device according to claim 14 , wherein the substitute object is determined by:
comparing the signatures of the at least two candidate objects with the signatures of the candidate substitute objects of the library;
comparing the candidate object criteria values of the at least two candidate objects, and
identifying the candidate substitute object of the library having a signature identical to or close to the signature of one of the at least two candidate objects, the criteria value of said one of the at least two candidate objects being higher than the criteria value of any other of the at least two candidate objects, the substitute object being the identified candidate substitute object of the library.
22. The device according to claim 14 , wherein the signatures of the candidate substitute objects of the library are pre-computed and stored in the library.