Aggregating images to generate content
An illustrative system constructs a map that associates pixel locations in a panoramic image to camera modules in a set of camera modules. The map may associate a first pixel location in the panoramic image to a first camera module in the set of camera modules and a second pixel location in the panoramic image to a second camera module in the set of camera modules. The second pixel location borders the first pixel location in the panoramic image. The system blends, based on at least one weight, a pixel value of the first pixel location captured by the first camera module with a pixel value of the second pixel location captured by the second camera module to determine a pixel value for a blended pixel in the panoramic image.
1. A method comprising:
constructing a map that associates a first pixel location in a panoramic image to a first camera module in a set of camera modules and a second pixel location in the panoramic image to a second camera module in the set of camera modules, the second pixel location bordering the first pixel location in the panoramic image;
blending, based on a weight, a pixel value of the first pixel location captured by the first camera module with a pixel value of the second pixel location captured by the second camera module to determine a pixel value for a blended pixel in the panoramic image; and
adjusting the weight based on a change in a viewing point of a user relative to a field of view boundary of the first camera module.
2. The method of claim 1 , wherein:
the weight is associated with the first camera module; and
the map indicates the weight is to be used to blend the pixel value of the first pixel location with a pixel value of a bordering pixel location.
3. The method of claim 1 , wherein:
the blending is further based on a second weight;
the second weight is associated with the second camera module; and
the map indicates the second weight is to be used to blend the pixel value of the second pixel location with a pixel value of a bordering pixel location.
4. The method of claim 1 , wherein the map indicates the weight.
5. The method of claim 1 , wherein the blending comprises:
separating the pixel value for the first pixel location into a first high-frequency part and a first low-frequency part;
separating the pixel value for the second pixel location into a second high-frequency part and a second low-frequency part;
combining, based on the weight, the first low-frequency part and the second low-frequency part to form a blended low-frequency part; and
selecting and combining the first high-frequency part or the second high-frequency part with the blended low-frequency part to determine the pixel value for the blended pixel.
6. The method of claim 1 , wherein the weight is for a low-frequency part of the first camera module.
7. The method of claim 1 , further comprising selecting the first camera module to be associated with the first pixel location in the map by:
determining the set of camera modules to have a point in their respective fields of view;
determining a viewing direction from the viewing point of the user to the point;
determining a set of camera viewing directions to the point for the set of camera modules, the set of camera viewing directions including a first viewing direction for the first camera module to the point; and
selecting the first camera module from the set of camera modules based on a camera viewing direction of the first camera module having a smaller angle with the viewing direction from the viewing point of the user to the point compared to other camera viewing directions associated with other camera modules in the set of camera modules.
8. The method of claim 1 , wherein adjusting the weight based on the change in the viewing point of the user relative to the field of view boundary of the first camera module comprises decreasing the weight as the viewing point of the user moves toward the field of view boundary of the first camera module.
9. A system comprising:
a processor; and
a non-transitory computer-readable medium communicatively coupled to the processor and storing instructions executable by the processor to perform operations comprising:
constructing a map that associates a first pixel location in a panoramic image to a first camera module in a set of camera modules and a second pixel location in the panoramic image to a second camera module in the set of camera modules, the second pixel location bordering the first pixel location in the panoramic image;
blending, based on a weight, a pixel value of the first pixel location captured by the first camera module with a pixel value of the second pixel location captured by the second camera module to determine a pixel value for a blended pixel in the panoramic image; and
adjusting the weight based on a change in a viewing point of a user relative to a field of view boundary of the first camera module.
10. The system of claim 9 , wherein:
the weight is associated with the first camera module; and
the map indicates the weight is to be used to blend the pixel value of the first pixel location with a pixel value of a bordering pixel location.
11. The system of claim 9 , wherein:
the blending is further based on a second weight;
the second weight is associated with the second camera module; and
the map indicates the second weight is to be used to blend the pixel value of the second pixel location with a pixel value of a bordering pixel location.
12. The system of claim 9 , wherein the map indicates the weight.
13. The system of claim 9 , wherein the blending comprises:
separating the pixel value for the first pixel location into a first high-frequency part and a first low-frequency part;
separating the pixel value for the second pixel location into a second high-frequency part and a second low-frequency part;
combining, based on the weight, the first low-frequency part and the second low-frequency part to form a blended low-frequency part; and
selecting and combining the first high-frequency part or the second high-frequency part with the blended low-frequency part to determine the pixel value for the blended pixel.
14. The system of claim 9 , wherein the weight is for a low-frequency part of the first camera module.
15. The system of claim 9 , wherein the instructions are further executable by the processor to perform an operation comprising selecting the first camera module to be associated with the first pixel location in the map by:
determining the set of camera modules to have a point in their respective fields of view;
determining a viewing direction from the viewing point of the user to the point;
determining a set of camera viewing directions to the point for the set of camera modules, the set of camera viewing directions including a first viewing direction for the first camera module to the point; and
selecting the first camera module from the set of camera modules based on a camera viewing direction of the first camera module having a smaller angle with the viewing direction from the viewing point of the user to the point compared to other camera viewing directions associated with other camera modules in the set of camera modules.
16. The system of claim 9 , wherein adjusting the weight based on the change in the viewing point of the user relative to the field of view boundary of the first camera module comprises decreasing the weight as the viewing point of the user moves toward the field of view boundary of the first camera module.
17. A non-transitory computer-readable medium storing instructions executable by a processor to cause the processor to:
construct a map that associates a first pixel location in a panoramic image to a first camera module in a set of camera modules and a second pixel location in the panoramic image to a second camera module in the set of camera modules, the second pixel location bordering the first pixel location in the panoramic image;
blend, based on a weight, a pixel value of the first pixel location captured by the first camera module with a pixel value of the second pixel location captured by the second camera module to determine a pixel value for a blended pixel in the panoramic image; and
adjust the weight based on a change in a viewing point of a user relative to a field of view boundary of the first camera module.
18. The non-transitory computer-readable medium of claim 17 , wherein:
the weight is associated with the first camera module; and
the map indicates the weight is to be used to blend the pixel value of the first pixel location with a pixel value of a bordering pixel location.
19. The non-transitory computer-readable medium of claim 17 , wherein:
the blending is further based on a second weight;
the second weight is associated with the second camera module; and
the map indicates the second weight is to be used to blend the pixel value of the second pixel location with a pixel value of a bordering pixel location.
20. The non-transitory computer-readable medium of claim 17 , wherein adjusting the weight based on the change in the viewing point of the user relative to the field of view boundary of the first camera module comprises decreasing the weight as the viewing point of the user moves toward the field of view boundary of the first camera module.