Alternate viewpoint rendering
View Patent ↗Described are computer-based methods and apparatuses, including computer program products, for alternate viewpoint rendering. A first image and a set of pixel-shift values for the pixels in the first image are received. A portion of a second image is generated, the second image comprising a different viewpoint than a viewpoint of the first image. A first pixel from the set of pixels is selected based on the different viewpoint. A first destination pixel is identified for the first pixel in the second image based on a pixel-shift value from the set of pixel-shift values associated with the first pixel. A value of the first destination pixel is determined based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel.
1. A non-transitory storage medium storing therein a stereoscopic image comprising a first image and a second image, the second image generated by the process of:
receiving, b a computing device, the first image comprising a set of pixels; and
generating, by the computing device, a portion of the second image, the second image comprising a different viewpoint than a viewpoint of the first image, comprising:
selecting, by the computing device, a first pixel from the set of pixels based on the different viewpoint;
identifying, by the computing device, a first destination pixel for the first pixel in the second image based on a pixel-shift value from a set of pixel-shift values associated with the first pixel, each pixel-shift value in the set of pixel-shift values comprising data indicative of a change in location of a corresponding pixel from the set of pixels from an original pixel location in the first image;
determining, by the computing device, a value of the first destination pixel based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.
2. The non-transitory storage medium of claim 1 , wherein the different viewpoint of the second image is one of:
left of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from loft to right; and
right of the viewpoint of the f IT Si Image, the method further comprising considering pixels in the set of pixels from the first image from right to left.
3. The non-transitory storage medium, of claim 1 , wherein determining the value of the first destination pixel comprises:
determining a pixel gap between the first destination pixel and a destination pixel of the previously considered pixel, the gap comprising a set of pixels; and
if the pixel gap is less than a threshold gap size, setting the pixel value for each pixel in the set of pixels of the pixel gap equal to the previously considered pixel.
4. The non-transitory storage medium of claim 1 comprising:
identifying a set of pixel gaps in the second image, wherein each pixel gap in the set of pixel gaps is greater than a threshold gap size; and
determining values for pixels in each pixel gap of the set of pixel gaps.
5. The non-transitory storage medium of claim 4 comprising:
selecting a pixel gap from the set of pixel gaps;
determining which of a left pixel horizontally adjacent to a left side of the pixel gap and a right pixel horizontally adjacent to a right side of the pixel gap has a lowest depth based on the different viewpoint; and
determining values for pixels in the pixel gap used on the lowest of the left pixel and the right pixel.
6. The non-transitory storage medium of claim 1 , further comprising:
receiving data indicative of a pixel shift value for the second image; and
shifting each pixel in the second image based on the pixel shift value.
7. The non-transitory storage medium of claim 1 , wherein the set of pixel-shift values are calculated based on depth information for the set of pixels of the first image and wherein the depth information comprises a depth matte comprising a depth value for each pixel in the first image.
8. A computerized method comprising:
receiving, by a computing device, the first image comprising a set of pixels; and
storing the first image and a second image in image data configured to be accessed by another device, the second image comprising a different viewpoint than a viewpoint of the first image and a portion of the second image is generated by the steps comprising:
selecting, b the computing, device, a first pixel from the set of pixels based on the different viewpoint;
identifying, by the computing device, a first destination pixel for the first pixel in the second image based on a pixel-shift value from a set of pixel-shift values associated with the first pixel, each pixel-shift value in the set of pixel-shift values comprising data indicative of a change in location of a corresponding pixel from the set of pixels from an original pixel location in the first image;
determining, by the computing device, a value of the first destination pixel based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.
9. The method of claim 8 , wherein determining the value of the first destination pixel comprises determining the value of the first destination pixel based on a value of the previously considered pixel and, a destination pixel for the previously considered pixel in the second image.
10. The method of claim 8 , wherein the different viewpoint of the second image is one of:
left of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from of to right; and
right of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from right to left.
11. The method of claim 8 , wherein determining the value of the first destination pixel comprises:
determining a pixel gap between the first destination pixel and a destination pixel of the previously considered pixel, the gap comprising a set of pixels; and
if the pixel gap is less than a threshold gap size, setting the pixel value for each pi in the set of pixels of the pixel gap equal to the previously considered pixel.
12. The method of claim 8 comprising:
identifying a set of pixel gaps in the second image, wherein each pixel gap in the set of pixel gaps is greater than a threshold gap size; and
determining values for pixels in each pixel gap of the set of pixel gaps.
13. The method of claim 12 comprising:
selecting a pixel gap from the set of pixel gaps;
determining which of a left pixel horizontally adjacent to a left side of the pixel gap and a right pixel horizontally adjacent to a right side of the pixel gap has a lowest depth based on the different viewpoint; and
determining values for pixels in the pixel gap based on the lowest of the left pixel and the right pixel.
14. The method of claim 8 , further comprising:
receiving data indicative of a pixel shift value for the second image; and
shifting each pixel in the second image based on the pixel shift value.
15. The method of claim 8 , wherein the set of pixel-shift values are calculated based on depth information for the set of pixels of the first image and wherein the depth information comprises a depth matte comprising a depth value for each pixel in the first image.
16. A computerized method comprising:
displaying, at a display device, a stereoscopic image comprising a first image and a second image, the second image comprising a different viewpoint than a viewpoint of the first image and generated by the steps of:
selecting, by the computing device, a first pixel from a set of pixels of the first image based on the different viewpoint;
identifying, by the computing device, a first destination pixel for the first pixel in the second image based on a pixel-shift value from a set of pixel-shift values associated with the first pixel, each pixel-shift value in the set of pixel-shift values comprising data indicative of a change in location of a corresponding pixel from the set of pixels from an original pixel location in the first image;
determining, by the computing device, a value of the first destination pixel based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.
17. The method of claim 16 , wherein the different viewpoint of the second image is one of:
left of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from left to right; and
right of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from right to left.
18. The method of claim 16 , wherein determining the value of the first destination pixel comprises:
determining a pixel gap between the first destination pixel and a destination pixel of the previously considered pixel, the gap comprising a sot of pixels; and
if the pixel gap is less than a threshold gap size, setting the pixel value for each pixel in the set of pixels of the pixel gap equal to the previously considered pixel.
19. The method of claim 16 comprising:
identifying a set of pixel gaps in the second image, wherein each pixel gap in the set of pixel gaps is greater than a threshold gap size, and determining values for pixels in each pixel gap of the set of pixel gaps.
20. The method of claim 19 comprising:
selecting a pixel gap from the set of pixel gaps;
determining which of a left pixel horizontally adjacent to a left side of the pixel gap and a right pixel horizontally adjacent to a right side of the pixel gap has a lowest depth based on the different viewpoint; and
determining values for pixels in the pixel gap based on the lowest of the left pixel and the right pixel.
21. The method of claim 16 further comprising:
receiving, data indicative of a pixel shift value for the second image; and
shifting each pixel in the second image based on the pixel shift value.
22. The method of claim 16 , wherein the set of pixel-shift values are calculated based on depth information for the set of pixels of the first image and wherein the depth information comprises a depth matte comprising a depth value for each pixel in the first image.
23. A computerized method comprising:
receiving, by a computing device, (a) a first image comprising a set of pixels and (b) a set of horizontal image translation data associated with the first image;
determining pixel-shift values for each pixel of the first image based on the horizontal image translation data and one or more control parameters; and
generating, by the computing device, a portion of a second image, the second image comprising a different viewpoint than a viewpoint of the first image, comprising:
selecting, by the computing device, a first pixel from the set of pixels based on the different viewpoint;
identifying, by the computing device, a first destination pixel for the first pixel in the second image based on a pixel-shift value from the set of pixel-shift values associated with the first pixel;
determining, by the computing device, a value of the first destination pixel based on at least one of (a) a previously considered pixel from the set of pixels of the first image that is horizontally adjacent to the first pixel or (b) the pixel-shift value associated with the first pixel;
determining the value of the first destination pixel was previously determined based on the previously considered pixel, the previously considered pixel having a larger shift than a shift of the first pixel;
not changing the value of the first destination pixel based on the first pixel;
selecting a second pixel that is horizontally adjacent to the first pixel and the previously considered pixel;
identifying a second destination pixel for the second pixel in the second image based on an associated pixel-shift value from the set of pixel-shift values, wherein the second destination pixel is horizontally adjacent to the first destination pixel; and
setting a value of the second destination pixel based on the first pixel, the second pixel, and the pixel-shift value associated with the second pixel.
24. The method of claim 23 , wherein the different viewpoint of the second image is one of:
left of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from left to right; and right of the viewpoint of the first image, the method further comprising considering pixels in the set of pixels from the first image from right to left.
25. The method of claim 23 , wherein determining the value of the first destination pixel comprises:
determining a pixel gap between the first destination pixel and a destination pixel of the previously considered pixel, the gap comprising a set of pixels; and
if the pixel gap is less than a threshold gap size, setting the pixel value for each pixel in the set of pixels of the pixel gap equal to the previously considered pixel.
26. The method of claim 23 comprising:
identifying a set of pixel gaps in the second image, wherein each pixel gap in the set of pixel gaps is greater than a threshold gap size; and
determining values for pixels in each pixel gap of the set of pixel gaps.
27. The method of claim 26 comprising:
selecting a pixel gap from the set of pixel gaps;
determining which of a left pixel horizontally adjacent to a left side of the pixel gap and a right pixel horizontally adjacent to a right side of the pixel gap has as lowest depth based on the different viewpoint; and
determining values for pixels in the pixel gap based on the lowest of the left pixel and the right pixel.
28. The method of claim 23 , further comprising:
receiving data indicative of a pixel shift value for the second image and shifting each pixel in the second image based on the pixel shift value.
29. The method of claim 23 , wherein the one or more control parameters includes depth information for the set of pixels: of the first image and wherein the depth information comprises a depth matte comprising a depth value for each pixel in the first image.