Method and device for blurring a virtual object in a video
In order to blur a virtual object in a video in real time as the video is acquired by a device capturing a real scene, a salient idea is used which estimates an apparent motion vector between two successive images, being captured at two successive device poses, in which the apparent motion vector estimation is based on a motion of the device. The successive images are then filtered based on the estimated apparent motion vector.
1. A method comprising:
obtaining an initial image of a video captured by a device at an initial device pose;
obtaining a current image of said video captured by the device at a current device pose;
estimating an apparent motion vector of a virtual object between said initial image and said current image, based on an angular rate associated with at least one motion sensor of said device moving from said initial device pose to said current device pose,
wherein said estimating is performed independent of any translation of said device moving from said initial device pose to said current device pose and independent of a distance between the virtual object and the current device pose; and
blurring the virtual object based on said apparent motion vector.
2. The method according to claim 1 , wherein said estimating said apparent motion vector further comprises estimating said current device pose, based on said initial device pose and on said angular rate.
3. The method according to claim 1 , further comprising estimating a current object location in said current image based on said apparent motion vector and on an initial object location in said initial image.
4. The method according to claim 3 , further comprising:
inserting said virtual object in said initial image, from a user interface, said inserting comprising associating a vertex of said virtual object with said initial object location in said initial image; and
estimating said initial device pose related to said initial object position corresponding to said initial object location.
5. The method according to claim 1 , wherein said virtual object has a trajectory, said method further comprising:
estimating a current object position based on said trajectory and an initial object position; and
estimating a current object location in said current image based on a projection of said current object position in said current image.
6. The method according to claim 5 , wherein estimating said apparent motion vector of said virtual object is based on a difference between said current object location in said current image and an initial object location in said initial image.
7. The method according to claim 1 , wherein said blurring said virtual object further comprises filtering the virtual object based on a spatial filter length and a spatial filter direction, said spatial filter direction corresponding to a direction of said apparent motion vector, and said spatial filter length being a factor of a module of said apparent motion vector, said spatial filter length corresponding to a number of neighboring pixels.
8. The method according to claim 7 , wherein said factor corresponds to a shutter aperture duration of said device.
9. The method according to claim 7 , wherein said filtering the virtual object further comprises:
obtaining a mask of said virtual object for said current image,
wherein the mask comprises non-null color values and a weight value of one for pixels of said current image corresponding to said virtual object;
spatially filtering said mask according to said spatial filter; and
blending the current image with the spatially filtered mask of the virtual object.
10. The method according to claim 1 , further comprising estimating said initial device pose based on a preceding image being captured by said device at a preceding device pose, said estimating said initial device pose being further based on said preceding device pose and a preceding motion of said device from said preceding device pose to said initial device pose.
11. A device comprising:
a processor configured to:
obtain an initial image of a video captured by a camera at an initial device pose;
obtain a current image of said video captured by the camera at a current device pose;
estimate an apparent motion vector of a virtual object between said initial image and said current image, based on an angular rate associated with at least one motion sensor indicating motion of the camera moving from said initial device pose to said current device pose,
wherein said estimating is performed independent of any translation of said device moving from said initial device pose to said current device pose and independent of a distance between the virtual object and the current device pose; and
blur the virtual object based on said apparent motion vector.
12. The device according to claim 11 , wherein the processor is further configured to estimate said current device pose, based on said initial device pose and on said angular rate.
13. The device according to claim 11 , further comprising the camera and the motion sensor.
14. The device according to claim 11 , wherein the processor is further configured to estimate a current object location in said current image based on said apparent motion vector and on an initial object location in said initial image.
15. The device according to claim 14 , wherein the processor is further configured to:
insert said virtual object in said initial image, from a user interface, said inserting comprising associating a vertex of said virtual object with said initial object location in said initial image; and
estimate said initial device pose related to said initial object position corresponding to said initial object location.
16. The device according to claim 11 , wherein the processor is further configured to estimate said initial device pose based on a preceding image being captured by said device at a preceding device pose, said estimating said initial device pose being further based on said preceding device pose and a preceding motion of said device from said preceding device pose to said initial device pose.
17. The device according to claim 11 ,
wherein the processor is further configured to obtain a mask of said virtual object for said current image, and
wherein the mask comprises non-null color values and a weight value of one for pixels of said current image corresponding to said virtual object, said virtual object being blurred based on said mask.
18. The device according to claim 17 , wherein the processor is further configured to filter the mask based on a spatial filter length and a spatial filter direction, said spatial filter direction corresponding to a direction of said apparent motion vector, and said spatial filter length being a factor of a module of said apparent motion vector, said spatial filter length corresponding to a number of neighboring pixels.
19. The device according to claim 18 , wherein the processor is further configured to blend the current image with the spatially filtered mask of the virtual object.
20. A non-transitory computer-readable storage medium comprising program code instructions executable by a processor for:
obtaining an initial image of a video captured by a device at an initial device pose;
obtaining a current image of said video captured by the device at a current device pose;
estimating an apparent motion vector of a virtual object between said initial image and said current image, based on an angular rate associated with at least one motion sensor of the device moving from said initial device pose to said current device pose,
wherein said estimating is performed independent of any translation of said device moving from said initial device pose to said current device pose and independent of a distance between the virtual object and the current device pose; and
blurring the virtual object based on said apparent motion vector.
21. The non-transitory computer-readable storage medium according to claim 20 , further comprising program code instructions executable by a processor for estimating a current object location in said current image based on said apparent motion vector and on an initial object location in said initial image.
22. The non-transitory computer-readable storage medium according to claim 21 , further comprising program code instructions executable by a processor for:
inserting said virtual object in said initial image, from a user interface, said inserting comprising associating a vertex of said virtual object with said initial object location in said initial image; and
estimating said initial device pose related to said initial object position corresponding to said initial object location.
23. The non-transitory computer-readable storage medium according to claim 20 , further comprising program code instructions executable by a processor for estimating said initial device pose based on a preceding image being captured by said device at a preceding device pose, said estimating said initial device pose being further based on said preceding device pose and a preceding motion of said device from said preceding device pose to said initial device pose.
24. The non-transitory computer-readable storage medium according to claim 20 , further comprising program code instructions executable by a processor for obtaining a mask of said virtual object for said current image, wherein the mask comprises non-null color values and a weight value of one for pixels of said current image corresponding to said virtual object, said virtual object being blurred based on said mask.
25. The non-transitory computer-readable storage medium according to claim 24 , further comprising program code instructions executable by a processor for filtering the mask based on a spatial filter length and a spatial filter direction, said spatial filter direction corresponding to a direction of said apparent motion vector, and said spatial filter length being a factor of a module of said apparent motion vector, said spatial filter length corresponding to a number of neighboring pixels.
26. The non-transitory computer-readable storage medium according to claim 25 , further comprising program code instructions executable by a processor for blending the current image with the spatially filtered mask of the virtual object.