User interface system, user interface method, and image editing device
Provided is a user interface system that includes a presentation means that presents information for receiving, from a user, designation related to a target to be subjected to color conversion by image editing for converting a color of an image using instance tracking and pixel tracking, and an acceptance means that receives the designation by the user.
1 . An information processing system, comprising:
circuitry configured to:
provide first information related to a region in a first frame of a first video, wherein
the region is a target of color conversion,
the color conversion of the region is based on an instance tracking method and a pixel tracking method across a plurality of temporally successive frames of the first video,
the color conversion of the region is propagated over time of the first video, and
the plurality of temporally successive frames includes the first frame of the first video;
receive, based on a first user input and the first information, second information associated with a designation of the region; and
control a screen to display:
the designation of the region; and
a result of the color conversion of the region that corresponds to a second video, wherein
a second frame from the displayed result is selected based on a second user input, and
the selected second frame is corrected.
2 . An information processing system, comprising:
circuitry configured to:
obtain information related to a target region in a first frame of a first video;
convert, based on the obtained information, color of the target region based on an instance tracking method and a pixel tracking method across a plurality of temporally successive frames of the first video, wherein
the conversion of the color of the target region is propagated over time of the first video, and
the plurality of temporally successive frames includes the first frame of the first video;
generate a result of the conversion of the color of the target region that corresponds to a second video;
control a screen to display the result of the conversion of the color of the target region;
select, based on a first user input, a second frame from the displayed result; and
correct the selected second frame.
3 . The information processing system according to claim 2 , wherein the circuitry is further configured to convert the color of the target region based on a model learned by machine learning.
4 . The information processing system according to claim 2 , wherein the conversion of the color is for one of
a black and white image, or
a color image.
5 . The information processing system according to claim 2 , wherein the circuitry is further configured to receive a reference image for the conversion of the color of the target region.
6 . The information processing system according to claim 2 , wherein the information is for identification of the target region.
7 . The information processing system according to claim 2 , wherein the information corresponds to a tracking ID of the target region.
8 . The information processing system according to claim 2 , wherein
the circuitry is further configured to receive a reference image for the conversion of the color of each target region of a plurality of target regions, and
the plurality of target regions includes the target region.
9 . The information processing system according to claim 2 , wherein the circuitry is further configured to receive a pixel designation for an image.
10 . The information processing system according to claim 9 , wherein the circuitry is further configured to receive a range of the image.
11 . The information processing system according to claim 2 , wherein the circuitry is further configured to correct, based on a second user input, the selected second frame.
12 . The information processing system according to claim 11 , wherein the circuitry is further configured to learn, based on information of the correction, a model by machine learning.
13 . The information processing system according to claim 2 , wherein the circuitry is further configured to generate information indicating a certainty factor related to the conversion of the color of the target region.
14 . The information processing system according to claim 13 , wherein
the generated information is for adjustment of the color, and
the adjustment is based on a second user input.
15 . An information processing system, comprising:
circuitry configured to:
obtain information related to a target region in a first frame of a first video;
perform object-level analysis to detect and track a specific object represented by the target region across a plurality of temporally successive frames of the first video;
perform pixel-level analysis to monitor changes in pixel values within the target region over a time period to maintain correspondence across the plurality of temporally successive frames, wherein the plurality of temporally successive frames includes the first frame of the first video;
identify and follow the target region across the plurality of temporally successive frames based on:
the obtained information,
the object-level analysis, and
the pixel-level analysis;
convert color of the target region based on the identification of the target region and the followed target region across the plurality of temporally successive frames, wherein
the conversion of the color of the target region is propagated over time of the first video;
generate a result of the conversion of the color of the target region that corresponds to a second video;
control a screen to display the result of the conversion of the color of the target region;
select, based on a first user input, a second frame from the displayed result; and
correct the selected second frame.