Image burst editing based on natural language processing (NLP) input

1 . A method comprising:

obtaining, using at least one processor of an electronic device, a sequence of images of a scene;

performing, using the at least one processor, instance segmentation of the sequence of images to generate a sequence of segmented images, wherein performing the instance segmentation comprises (i) segmenting each of the images into semantic classes and (ii) separately identifying two or more objects in the scene associated with a common semantic class, wherein different semantic classes are associated with different types of image contents, wherein each identified object is associated with (i) a first bounding box around the identified object in the sequence of segmented images and (ii) a more-precise boundary of the identified object within the first bounding box; and

processing, using the at least one processor, the sequence of segmented images to generate a final image of the scene, the sequence of segmented images processed using one or more image processing operations determined using a natural language processing (NLP) engine;

wherein processing the sequence of segmented images to generate the final image of the scene comprises:

identifying at least one second bounding box around at least one specific object in the sequence of images of the scene;

identifying at least one of the first bounding boxes that overlaps with the at least one second bounding box;

identifying the more-precise boundary associated with each of the at least one of the first bounding boxes; and

performing the one or more image processing operations using the more-precise boundary associated with each of the at least one of the first bounding boxes.

2 . The method of claim 1 , further comprising:

determining the one or more image processing operations to be performed using the NLP engine.

3 . The method of claim 2 , wherein determining the one or more image processing operations to be performed comprises:

providing a user input to the NLP engine;

identifying an action to be performed from among multiple actions based on the user input using the NLP engine;

identifying the at least one specific object in the scene based on the user input using the NLP engine; and

identifying the one or more image processing operations to be performed in order to perform the identified action involving the at least one specific object using the NLP engine.

4 . The method of claim 3 , wherein:

determining the one or more image processing operations to be performed further comprises mapping the at least one specific object to results of the instance segmentation; and

the one or more image processing operations to be performed are identified in order to perform the identified action involving the at least one specific object as mapped to the results of the instance segmentation.

5 . The method of claim 3 , wherein identifying the at least one specific object in the scene based on the user input comprises using machine learning-based dependency parsing to identify the at least one specific object in the scene as being associated with the identified action.

6 . The method of claim 3 , wherein determining the one or more image processing operations to be performed further comprises determining an order in which to perform multiple image processing operations.

7 . The method of claim 1 , wherein the one or more image processing operations are performed in order to at least one of:

remove the at least one specific object in the scene;

create artistic motion blurring with motion that is extrapolation based on the sequence of images;

enhance colors of the at least one specific object in the scene;

change a color saturation in the sequence of images;

remove colors from the at least one specific object in the scene; or

stretch or shrink the at least one specific object in the scene.

8 . An electronic device comprising:

at least one imaging sensor configured to capture a sequence of images of a scene; and

at least one processor configured to:

perform instance segmentation of the sequence of images to generate a sequence of segmented images, wherein, to perform the instance segmentation, the at least one processor is configured to (i) segment each of the images into semantic classes and (ii) separately identify two or more objects in the scene associated with a common semantic class, wherein different semantic classes are associated with different types of image contents, wherein each identified object is associated with (i) a first bounding box around the identified object in the sequence of segmented images and (ii) a more-precise boundary of the identified object within the first bounding box; and

process the sequence of segmented images to generate a final image of the scene, wherein the at least one processor is configured to process the sequence of segmented images using one or more image processing operations determined using a natural language processing (NLP) engine;

wherein, to process the sequence of segmented images to generate the final image of the scene, at least one processor is configured to:

identify at least one second bounding box around at least one specific object in the sequence of images of the scene;

identify at least one of the first bounding boxes that overlaps with the at least one second bounding box;

identify the more-precise boundary associated with each of the at least one of the first bounding boxes; and

perform the one or more image processing operations using the more-precise boundary associated with each of the at least one of the first bounding boxes.

9 . The electronic device of claim 8 , wherein the at least one processing device processor is further configured to determine the one or more image processing operations to be performed using the NLP engine.

10 . The electronic device of claim 9 , wherein, to determine the one or more image processing operations to be performed, the at least one processor is configured to:

provide a user input to the NLP engine;

identify an action to be performed from among multiple actions based on the user input using the NLP engine;

identify the at least one specific object in the scene based on the user input using the NLP engine; and

identify the one or more image processing operations to be performed in order to perform the identified action involving the at least one specific object using the NLP engine.

11 . The electronic device of claim 10 , wherein, to determine the one or more image processing operations to be performed, the at least one processor is further configured to map the at least one specific object to results of the instance segmentation.

12 . The electronic device of claim 10 , wherein the at least one processor is configured to use machine learning-based dependency parsing to identify the at least one specific object in the scene as being associated with the identified action.

13 . The electronic device of claim 10 , wherein the at least one processor is further configured to determine an order in which to perform multiple image processing operations.

14 . The electronic device of claim 8 , wherein the at least one processor is configured to perform the one or more image processing operations in order to at least one of:

remove the at least one specific object in the scene;

create artistic motion blurring with motion that is extrapolation based on the sequence of images;

enhance colors of the at least one specific object in the scene;

change a color saturation in the sequence of images;

remove colors from the at least one specific object in the scene; or

stretch or shrink the at least one specific object in the scene.

15 . A non-transitory machine readable medium containing instructions that when executed cause at least one processor of an electronic device to:

obtain a sequence of images of a scene;

perform instance segmentation of the sequence of images to generate a sequence of segmented images, wherein the instructions that when executed cause the at least one processor to perform the instance segmentation comprise instructions that when executed cause the at least one processor to (i) segment each of the images into semantic classes and (ii) separately identify two or more objects in the scene associated with a common semantic class, wherein different semantic classes are associated with different types of image contents, wherein each identified object is associated with (i) a first bounding box around the identified object in the sequence of segmented images and (ii) a more-precise boundary of the identified object within the first bounding box; and

process the sequence of segmented images to generate a final image of the scene, wherein the instructions that when executed cause the at least one processor to process the sequence of segmented images comprise instructions that when executed cause the at least one processor to process the sequence of segmented images using one or more image processing operations determined using a natural language processing (NLP) engine;

wherein the instructions that when executed cause the at least one processor to process the sequence of segmented images to generate the final image of the scene comprise instructions that when executed cause the at least one processor to:

identify at least one second bounding box around at least one specific object in the sequence of images of the scene;

identify at least one of the first bounding boxes that overlaps with the at least one second bounding box;

identify the more-precise boundary associated with each of the at least one of the first bounding boxes; and

perform the one or more image processing operations using the more-precise boundary associated with each of the at least one of the first bounding boxes.

16 . The non-transitory machine readable medium of claim 15 , further containing instructions that when executed cause the at least one processor to determine the one or more image processing operations to be performed using the NLP engine.

17 . The non-transitory machine readable medium of claim 16 , wherein the instructions that when executed cause the at least one processor to determine the one or more image processing operations to be performed comprise:

instructions that when executed cause the at least one processor to:

provide a user input to the NLP engine;

identify an action to be performed from among multiple actions based on the user input using the NLP engine;

identify the at least one specific object in the scene based on the user input using the NLP engine; and

identify the one or more image processing operations to be performed in order to perform the identified action involving the at least one specific object using the NLP engine.

18 . The non-transitory machine readable medium of claim 17 , wherein the instructions that when executed cause the at least one processor to determine the one or more image processing operations to be performed further comprise:

instructions that when executed cause the at least one processor to map the at least one specific object to results of the instance segmentation.

19 . The non-transitory machine readable medium of claim 17 , further containing instructions that when executed cause the at least one processor to determine an order in which to perform multiple image processing operations.

20 . The non-transitory machine readable medium of claim 15 , wherein the instructions when executed cause the at least one processor to perform the one or more image processing operations in order to at least one of:

remove the at least one specific object in the scene;

create artistic motion blurring with motion that is extrapolation based on the sequence of images;

enhance colors of the at least one specific object in the scene;

change a color saturation in the sequence of images;

remove colors from the at least one specific object in the scene; or

stretch or shrink the at least one specific object in the scene.