Alternate viewpoint rendering
View Patent ↗Described are computer-based methods and apparatuses, including computer program products, for alternate viewpoint rendering. A first image and a set of pixel-shift values for the pixels in the first image are received. A portion of a second image is generated, the second image comprising a different viewpoint than a viewpoint of the first image. A first pixel from the set of pixels is selected based on the different viewpoint. A first destination pixel is identified for the first pixel in the second image based on a pixel-shift value from the set of pixel-shift values associated with the first pixel. A value of the first destination pixel is determined based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel.
1. A computerized method comprising:
receiving, by a computing device, (a) a first image comprising a set of pixels and (b) a set of pixel-shift values for the set of pixels, each pixel-shift value comprising data indicative of a change in location of a corresponding pixel from the set of pixels from an original pixel location in the first image; and
receiving, by an input unit of a computer, a first two dimensional image and a second two dimensional image, each two dimensional image comprising a pixel size;
generating, by the computing device, a portion of a second image, the second image comprising a different viewpoint than a viewpoint of the first image, comprising:
selecting, by the computing device, a first pixel from the set of pixels based on the different viewpoint;
identifying, by the computing device, a first destination pixel for the first pixel in the second image based on a pixel-shift value from the set of pixel-shift values associated with the first pixel;
determining, by the computing device, a value of the first destination pixel based on at least one of(a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.
2. The method of claim 1 , wherein determining comprises determining the value of the first destination pixel based on a value of the previously considered pixel and a destination pixel for the previously considered pixel in the second image.
3. The method of claim 1 , wherein the different viewpoint of the second image is left of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from left to right.
4. The method of claim 1 , wherein the different viewpoint of the second image is right of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from right to left.
5. The method of claim 1 , wherein determining the value of the first destination pixel comprises:
determining a pixel gap between the first destination pixel and a destination pixel of the previously considered pixel, the gap comprising a set of pixels; and
if the pixel gap is less than a threshold gap size, setting the pixel value for each pixel in the set of pixels of the pixel gap equal to the previously considered pixel.
6. The method of claim 1 , wherein:
identifying the first destination pixel comprises identifying the first destination pixel based on an integer portion of the pixel-shift value associated with the first pixel; and
determining the value of the first destination pixel comprises determining the value based on (a) both the first pixel and the previously considered pixel and (b) a decimal portion of the pixel-shift value associated with the first pixel.
7. The method of claim 1 , wherein the pixel-shift value associated with the first pixel is an integer number, and determining comprises setting the value of the first destination pixel equal to the first pixel.
8. The method of claim 1 , wherein the previously considered pixel is closer to a viewer than the first pixel such that the previously considered pixel obscures the first pixel from the viewer.
9. The method of claim 1 comprising:
identifying a set of pixel gaps in the second image, wherein each pixel gap in the set of pixel gaps is greater than a threshold gap size; and
determining values for pixels in each pixel gap of the set of pixel gaps.
10. The method of claim 9 comprising:
selecting a pixel gap from the set of pixel gaps;
determining which of a left pixel horizontally adjacent to a left side of the pixel gap and a right pixel horizontally adjacent to a right side of the pixel gap has a lowest depth based on the different viewpoint; and
determining values for pixels in the pixel gap based on the lowest of the left pixel and the right pixel.
11. The method of claim 1 , further comprising:
receiving data indicative of a pixel shift value for the second image; and
shifting each pixel in the second image based on the pixel shift value.
12. The method of claim 1 , wherein the set of pixel-shift values are calculated based on depth information for the set of pixels of the first image.
13. The method of claim 12 , wherein the depth information comprises a depth matte comprising a depth value for each pixel in the first image.
14. An apparatus comprising a processor and memory configured to:
receive (a) a first image comprising a set of pixels and (b) a set of pixel-shift values for the set of pixels, each pixel-shift value comprising data indicative of a change in location of a corresponding pixel from the set of pixels from an original pixel location in the first image; and
generate a portion of a second image, the second image comprising a different viewpoint than a viewpoint of the first image, comprising:
selecting a first pixel from the set of pixels based on the different viewpoint;
identifying a first destination pixel for the first pixel in the second image based on a pixel-shift value from the set of pixel-shift values associated with the first pixel; and
determining a value of the first destination pixel based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.
15. A computer program product, tangibly embodied in a non-transitory computer readable medium, the computer program product including instructions being configured to cause a data processing apparatus to:
receive (a) a first image comprising a set of pixels and (b) a set of pixel-shift values for the set of pixels, each pixel-shift value comprising data indicative of a change in location of a corresponding pixel from the set of pixels from an original pixel location in the first image; and
generate a portion of a second image, the second image comprising a different viewpoint than a viewpoint of the first image, comprising:
selecting a first pixel from the set of pixels based on the different viewpoint;
identifying a first destination pixel for the first pixel in the second image based on a pixel-shift value from the set of pixel-shift values associated with the first pixel; and
determining a value of the first destination pixel based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.
16. An apparatus comprising a means for:
receiving (a) a first image comprising a set of pixels and (b) a set of pixel-shift values for the set of pixels, each pixel-shift value comprising data indicative of a change in location of a corresponding pixel from the set of pixels from an original pixel location in the first image; and
generating a portion of a second image, the second image comprising a different viewpoint than a viewpoint of the first image, comprising:
selecting a first pixel from the set of pixels based on the different viewpoint;
identifying a first destination pixel for the first pixel in the second image based on a pixel-shift value from the set of pixel-shift values associated with the first pixel; and
determining a value of the first destination pixel based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.