Method for real-time video processing involving changing features of an object in the video
A method for real-time video processing for changing features of an object in a video, the method comprises: providing an object in the video, the object being at least partially and at least occasionally presented in frames of the video; detecting the object in the video; generating a list of at least one element of the object, the list being based on the object's features to be changed according to a request for modification; detecting the at least one element of the object in the video; tracking the at least one element of the object in the video; and transforming the frames of the video such that the at least one element of the object is modified according to the request for modification.
1. A method, comprising:
providing at least a portion of an object in frames of a video;
detecting the object in the video, the object including a set of features;
generating a list of at least one element of the object, the list being based on the set of features;
transforming at least one feature of the object within the frames of the video to generate modified frames with at least one modified feature, the at least one feature associated with an element selected from the list and the at least one feature being transformed in the modified frames within the video in which the object is detected while the video is provided at a computing device and the object is detected in the video; and
providing the modified frames including the at least one modified feature within the video.
2. The method of claim 1 , wherein transforming the at least one feature further comprises:
generating a mesh based on one or more characteristic points of the object; and
transforming the at least one feature based on the mesh and the one or more characteristic points of the object.
3. The method of claim 2 , wherein generating the mesh further comprises:
generating a first set of points on the mesh for characteristic points associated with the at least one element of the object;
generating a second set of points on the mesh based on the set of first points and a modification to be applied in generating the at least one modified feature; and
transforming the frames of the video such that the at least one feature of the object based on the second set of points on the mesh.
4. The method of claim 3 , further comprising:
receiving a request for modification representing a modification to be applied to the at least one feature; and
generating the second set of points on the mesh based on the request for modification and the set of first points.
5. The method of claim 1 , further comprising:
identifying an area on the object in the video, the area corresponding to the at least one feature of the object; and
transforming the area on the object within the frames of the video to generate modified frames with at least one modified area.
6. The method of claim 5 , further comprising:
generating a mesh based on one or more characteristic points of the object;
generating a first set of points on the mesh for characteristic points associated with the at least one element of the object and identifying the area based on the first set of points generated on the mesh; and
transforming the area on the object within the frames of the video based on the first set of points on the mesh.
7. The method of claim 1 , further comprising:
generating a mesh based on one or more characteristics points of the object;
generating a grid associated with a background of the video; and
transforming the at least one feature based on the mesh and the one or more characteristic points of the object while maintaining the background of the video based on the grid.
8. A system, comprising:
one or more processors; and
a non-transitory processor-readable storage medium storing processor executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
providing at least a portion of an object in frames of a video;
detecting the object in the video, the object including a set of features;
generating a list of at least one element of the object, the list being based on the set of features;
transforming at least one feature of the object within the frames of the video to generate modified frames with at least one modified feature, the at least one feature associated with an element selected from the list and the at least one feature being transformed in the modified frames within the video in which the object is detected while the video is provided at a computing device and the object is detected in the video; and
providing the modified frames including the at least one modified feature within the video.
9. The system of claim 8 , wherein transforming the at least one feature further comprises:
generating a mesh based on one or more characteristic points of the object; and
transforming the at least one feature based on the mesh and the one or more characteristic points of the object.
10. The system of claim 9 , wherein generating the mesh further comprises:
generating a first set of points on the mesh for characteristic points associated with the at least one element of the object;
generating a second set of points on the mesh based on the set of first points and a modification to be applied in generating the at least one modified feature; and
transforming the frames of the video such that the at least one feature of the object based on the second set of points on the mesh.
11. The system of claim 10 , wherein the operations further comprise:
receiving a request for modification representing a modification to be applied to the at least one feature; and
generating the second set of points on the mesh based on the request for modification and the set of first points.
12. The system of claim 8 , wherein the operations further comprise:
identifying an area on the object in the video, the area corresponding to the at least one feature of the object; and
transforming the area on the object within the frames of the video to generate modified frames with at least one modified area.
13. The system of claim 12 , wherein the operations further comprise:
generating a mesh based on one or more characteristic points of the object;
generating a first set of points on the mesh for characteristic points associated with the at least one element of the object and identifying the area based on the first set of points generated on the mesh; and
transforming the area on the object within the frames of the video based on the first set of points on the mesh.
14. The system of claim 8 , wherein the operations further comprise:
generating a mesh based on one or more characteristics points of the object;
generating a grid associated with a background of the video; and
transforming the at least one feature based on the mesh and the one or more characteristic points of the object while maintaining the background of the video based on the grid.
15. A non-transitory processor-readable storage medium storing processor executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising:
providing at least a portion of an object in frames of a video;
detecting the object in the video, the object including a set of features;
generating a list of at least one element of the object, the list being based on the set of features;
transforming at least one feature of the object within the frames of the video to generate modified frames with at least one modified feature, the at least one feature associated with an element selected from the list and the at least one feature being transformed in the modified frames within the video in which the object is detected while the video is provided at a computing device and the object is detected in the video; and
providing the modified frames including the at least one modified feature within the video.
16. The non-transitory processor-readable storage medium of claim 15 , wherein transforming the at least one feature further comprises:
generating a mesh based on one or more characteristic points of the object; and
transforming the at least one feature based on the mesh and the one or more characteristic points of the object.
17. The non-transitory processor-readable storage medium of claim 16 , wherein generating the mesh further comprises:
generating a first set of points on the mesh for characteristic points associated with the at least one element of the object;
generating a second set of points on the mesh based on the set of first points and a modification to be applied in generating the at least one modified feature; and
transforming the frames of the video such that the at least one feature of the object based on the second set of points on the mesh.
18. The non-transitory processor-readable storage medium of claim 17 , wherein the operations further comprise:
receiving a request for modification representing a modification to be applied to the at least one feature; and
generating the second set of points on the mesh based on the request for modification and the set of first points.
19. The non-transitory processor-readable storage medium of claim 15 , wherein the operations further comprise:
identifying an area on the object in the video, the area corresponding to the at least one feature of the object; and
transforming the area on the object within the frames of the video to generate modified frames with at least one modified area.
20. The non-transitory processor-readable storage medium of claim 15 , wherein the operations further comprise:
generating a mesh based on one or more characteristics points of the object;
generating a grid associated with a background of the video; and
transforming the at least one feature based on the mesh and the one or more characteristic points of the object while maintaining the background of the video based on the grid.