Scalable massive parallelization of overlapping patch aggregation
Techniques for enhancing an image using pixel-specific processing are disclosed. An image can be enhanced by updating certain pixels through patch aggregation. Neighboring pixels of a selected pixel are identified. Respective patch values for patches containing the selected pixel are determined. Patch values provide update information for updating the respective pixels in the patch. Relevant patch values for the selected pixel are identified by identifying associated patches of the pixel. Information from the relevant patch values of the selected pixel may be obtained. Using this information, pixel-specific processing may be performed to determine an updated pixel value for the selected pixel or for neighboring pixels of the selected pixel. Pixel-specific processes may be executed for each of the selected or neighboring pixels. These pixel-specific processes can be executed in parallel. Therefore, through the execution of pixel-specific processes, which may be performed concurrently, an enhanced image may be determined.
1. A method comprising:
receiving an image comprising pixels located in the image, each pixel having a pixel value indicative of how the pixel is displayed in the image;
determining a first subset of the pixels located in the image;
determining a second subset of pixels located in the image, wherein at least one pixel in the first subset neighbors at least one pixel in the second subset; and
for each pixel in the first subset:
determining multiple overlapping patches within the image, the multiple patches containing the pixel,
using an image enhancement algorithm to determine a respective patch value for each patch of the multiple overlapping patches, each patch value providing update information for updating the respective patch containing the pixel,
identifying relevant patch values of the overlapping patches that provide information for updating the respective patch containing the pixel, and
performing a first pixel-specific thread process by a processor to determine an aggregate patch value based on the relevant patch values; and
performing a second pixel-specific thread process by the processor to update, based on the aggregate patch value for at least one pixel from the first subset, the pixel value for a neighboring pixel in the second subset.
2. The method of claim 1 further comprising performing a first plurality of pixel-specific thread processes executed in parallel to determine aggregate patch values based on respective patch values relevant to each respective pixel in the first subset.
3. The method of claim 1 further comprising performing a second plurality of pixel-specific thread processes executed in parallel to update, based on the respective aggregate patch values for the at least one pixel in the first subset, the respective pixel values for the respective neighboring pixels in the second subset.
4. The method of claim 1 , wherein the aggregate patch value is determined based on averaging the relevant patch values.
5. The method of claim 1 , wherein the aggregate patch value is determined based on a maximum relevant patch value or a minimum relevant patch value.
6. The method of claim 1 , further comprising determining a weight for each relevant patch value based on the pixel in the first subset being located within a respective patch for the relevant patch value; and wherein updating the pixel value of the neighboring pixel in the second subset is further based on the respective weight of each relevant patch value.
7. The method of claim 1 further comprising performing additional iterations of updating the pixel value for the neighboring pixel in the second subset, wherein, in each of the additional iterations, a previous image from the prior iteration is used as the input image.
8. The method of claim 1 , wherein patch values are stored in a look-up table.
9. The method of claim 1 , wherein information of patches corresponding to the relevant patch values for each respective pixel in the first subset is kept in a look-up table.
10. The method of claim 1 , wherein relevant patch values are identified from respective patches that contain each respective pixel in the first subset.
11. A system comprising:
one or more processor for executing an image editing module comprising computer-readable instructions, which, when executed, cause the one or more processor to perform operations comprising:
receiving an image comprising pixels located in the image, each pixel having a pixel value indicative of how the pixel is displayed in the image;
determining a first subset of the pixels located in the image;
determining a second subset of pixels located in the image, wherein at least one pixel in the first subset neighbors at least one pixel in the second subset; and
for each pixel in the first subset:
determining multiple overlapping patches within the image, the multiple overlapping patches containing the pixel,
using an image enhancement algorithm to determine a respective patch value for each patch of the multiple overlapping patches, each patch value providing update information for updating the respective patch containing the pixel,
identifying relevant patch values of the overlapping patches that provide information for updating the respective patch containing the pixel, and
performing a first pixel-specific thread process to determine an aggregate patch value for the pixel, based on the relevant patch values; and
performing a second pixel-specific thread process by the processor to update, based on the aggregate patch value for at least one pixel from the first subset, the pixel value for a neighboring pixel in the second subset.
12. The system of claim 11 , wherein the image editing module comprises further instructions for:
performing a first plurality of pixel-specific thread processes executed in parallel to determine aggregate patch values for multiple pixels based on respective patch values relevant to each respective pixel in the first subset.
13. The system of claim 11 , wherein the image editing module comprises further instructions for:
performing a second plurality of pixel-specific thread processes executed in parallel to update, based on the respective aggregate patch values for the at least one pixel in the first subset, the respective pixel values for the respective neighboring pixels in the second subset.
14. The system of claim 11 , wherein the image editing module comprises further instructions for:
determining a weight for each relevant patch value based on the pixel in the first subset being located within a respective patch for the relevant patch value; and wherein updating the pixel value of the neighboring pixel in the second subset is further based on the respective weight of each relevant patch value.
15. A non-transitory computer-readable medium tangibly embodying program code executable by at least one processor for providing an image editing application, the program code comprising:
program code for receiving an image comprising pixels located in the image, each pixel having a pixel value indicative of how the pixel is displayed in the image;
program code for determining a first subset of the pixels located in the image;
program code for determining a second subset of pixels located in the image, wherein at least one pixel in the first subset neighbors at least one pixel in the second subset; and
for each pixel in the first subset:
program code for determining multiple overlapping patches within the image, the multiple overlapping patches containing the pixel,
program code for using an image enhancement algorithm to determine a respective patch value for each patch of the multiple overlapping patches, each patch value providing update information for updating the respective patch containing the pixel,
program code for identifying relevant patch values of the overlapping patches that provide information for updating the respective patch containing the pixel, and
program code for performing a first pixel-specific thread process to determine an aggregate patch value for the pixel, based on the relevant patch; and
performing a second pixel-specific thread process by the processor to update, based on the aggregate patch value for at least one pixel from the first subset, the pixel value for a neighboring pixel in the second subset.
16. The non-transitory computer-readable medium of claim 15 , further comprising:
program code for performing a first plurality of pixel-specific thread processes executed in parallel to determine aggregate patch values based on respective patch values relevant to each respective pixel in the first subset.
17. The non-transitory computer-readable medium of claim 15 , further comprising:
program code for performing a second plurality of pixel-specific thread processes executed in parallel to update, based on the respective aggregate patch values for the at least one pixel in the first subset, the respective pixel values for the respective neighboring pixels in the second subset.
18. The non-transitory computer-readable medium of claim 15 , further comprising:
program code for determining a weight for each relevant patch value based on the pixel in the first subset being located within a respective patch for the relevant patch value; and wherein updating the pixel value of the neighboring pixel is further based on the respective weight of each relevant patch value.
19. The non-transitory computer-readable medium of claim 15 , wherein the aggregate patch value is determined based on averaging the relevant patch values.
20. The non-transitory computer-readable medium of claim 15 , wherein the aggregate patch value is determined based on a maximum relevant patch value or a minimum relevant patch value.