Creating multi-dimensional object representations

1. A computer-implemented method comprising:

under the control of one or more computer systems configured with executable instructions,

capturing a background image for each of a plurality of cameras, a background image portraying a background;

capturing a plurality of object images, including at least one object image for each of the plurality of cameras, an object image portraying a viewpoint of an object against the background;

creating a difference image by subtracting the background image of the viewpoint from the at least one object image of the viewpoint;

determining high confidence pixels, the high confidence pixels being pixels that exceed a first threshold contrast with background image;

determining low confidence pixels, the low confidence pixels being pixels that exceed a second threshold contrast with the background image, the second threshold contrast being lower than the first threshold contrast;

determining pixels associated with the object, including high confidence pixels and a subset of low confidence pixels; and

creating a plurality of segmentation masks corresponding to the plurality of object images, a segmentation mask including (i) filled space, comprising pixels associated with the object, and (ii) unfilled space, comprising pixels not associated with the object.

2. The computer-implemented method of claim 1 , further comprising:

determining object contours in the at least one object image, wherein the subset of low confidence pixels includes low confidence pixels that can be connected to a high confidence pixel without crossing an object contour;

wherein determining high confidence pixels includes creating a high confidence image mask based at least in part on the difference image, the high confidence image mask indicating pixels that exceed the first threshold contrast with background image; and

wherein determining low confidence pixels includes creating a low confidence image mask based at least in part on the difference image, the low confidence image mask indicating pixels that exceed the second threshold contrast with background image.

3. The computer-implemented method of claim 1 , further comprising:

creating an object mesh structure, based at least in part on the plurality of segmentations masks, the object mesh structure representing the object in three dimensions;

receiving a request to display a representation of the object from a specified viewpoint;

sending the object mesh structure, the plurality of segmentation masks, and the plurality of object images to a client device; and

displaying the object image portraying the specified viewpoint overlaid on the object mesh structure oriented to the specified viewpoint.

4. The computer-implemented method of claim 3 , further comprising:

determining a ground plane intersecting the object mesh structure;

determining a portion of the object mesh structure below the ground plane; and

removing from the object mesh structure the portion of the object mesh structure below the ground plane.

5. The computer-implemented method of claim 3 , further comprising:

projecting the object mesh structure onto a segmentation mask of the plurality of segmentation masks; and

refining the segmentation mask based at least in part on the projected object mesh structure.

6. A computer-implemented method comprising:

under the control of one or more computer systems configured with executable instructions,

determining an object image and background image, the background image including a background, the object image including an object in front of the background and representing a viewpoint of the object;

determining high confidence pixels, the high confidence pixels being pixels that exceed a first threshold contrast with background image;

determining low confidence pixels, the low confidence pixels being pixels that exceed a second threshold contrast with the background image, the second threshold contrast being lower than the first threshold contrast;

determining pixels associated with the object, including high confidence pixels and a subset of low confidence pixels; and

creating a segmentation mask based in part on the object image, the segmentation mask including (i) filled space, comprising pixels associated with the object, and (ii) unfilled space, comprising pixels not associated with the object.

7. The computer-implemented method of claim 6 , further comprising:

creating a plurality of segmentation masks from a respective plurality of viewpoints of the object;

creating an object mesh structure based at least in part on the plurality of segmentation masks, the object mesh structure representing the object in three dimensions;

receiving a request to display a three-dimensional representation of the object from a specified viewpoint; and

displaying the object mesh structure from the specified viewpoint overlaid by the object image portraying the specified viewpoint.

8. The computer-implemented method of claim 7 , further comprising

projecting the object mesh structure from a viewpoint to a segmentation mask associated with the viewpoint; and

refining the segmentation mask based at least in part on projecting the object mesh structure.

9. The computer-implemented method of claim 7 , further comprising:

determining a ground plane intersecting the object mesh structure;

determining a portion of the object mesh structure below the ground plane; and

removing from the object mesh structure the portion of the object mesh structure below the ground plane.

10. The computer-implemented method of claim 6 , further comprising:

determining object contours in the at least one object image, wherein the subset of low confidence pixels includes low confidence pixels that can be connected to a high confidence pixel without crossing an object contour;

wherein determining high confidence pixels includes creating a high confidence image mask based at least in part on subtracting the background image from the object image, the high confidence image mask indicating pixels that exceed the first threshold contrast with background image; and

wherein determining low confidence pixels includes creating a low confidence image mask based at least in part on subtracting the background image from the object image, the low confidence image mask indicating pixels that exceed the second threshold contrast with background image.

11. The computer-implemented method of claim 10 , wherein determining object contours of the object comprises:

creating an edge map using an edge detection algorithm, the edge map including one or more gaps;

using a close morphological operation to close the gaps in the edge map;

traversing the edge map from an image boundary;

determining that edge pixels first encountered when traversing the edge map from an image boundary are contour pixels; and

determining object contours based at least in part on determining contour pixels.

12. A system comprising:

at least one processor;

memory including instructions that, when executed by the at least one processor, cause the system to:

determine a background image, the background image including a background;

determine an object image, the object image including an object in front of the background and representing a viewpoint of the object;

determining high confidence pixels, the high confidence pixels being pixels that exceed a first threshold contrast with background image;

determining low confidence pixels, the low confidence pixels being pixels that exceed a second threshold contrast with the background image, the second threshold contrast being lower than the first threshold contrast;

determining pixels associated with the object, including high confidence pixels and a subset of low confidence pixels; and

create a segmentation mask based in part on the object image, the segmentation mask including (i) filled space, comprising pixels associated with the object, and (ii) unfilled space, comprising pixels not associated with the object.

13. The system of claim 12 , wherein the memory further includes instructions that, when executed by the at least one processor, cause the system to:

create a plurality of segmentation masks from a respective plurality of viewpoints of the object; and

create an object mesh structure based at least in part on the plurality of segmentation masks, the object mesh structure representing the object in three dimensions;

receive a request to display a three-dimensional representation of the object from a specified viewpoint; and

display the object mesh structure from the specified viewpoint overlaid by the object image portraying the specified viewpoint.

14. The system of claim 13 , wherein the memory further includes instructions that, when executed by the at least one processor, cause the system to:

determine object contours in the at least one object image, wherein the subset of low confidence pixels includes low confidence pixels that can be connected to a high confidence pixel without crossing an object contour;

wherein determining high confidence pixels includes creating a high confidence image mask based at least in part on subtracting the background image from the object image, the high confidence image mask indicating pixels that exceed the first threshold contrast with background image; and

wherein determining low confidence pixels includes creating a low confidence image mask based at least in part on subtracting the background image from the object image, the low confidence image mask indicating pixels that exceed the second threshold contrast with background image.

15. The system of claim 13 , wherein creating the object mesh structure comprises:

determining a three-dimensional object environment; and

subtracting the unfilled space of each segmentation mask from the three-dimensional object environment.

16. The system of claim 14 , wherein determining object contours of the object comprises:

creating an edge map using an edge detection algorithm, the edge map including one or more gaps;

using a close morphological operation to close the gaps in the edge map;

traversing the edge map from an image boundary;

determining that edge pixels first encountered when traversing the edge map from an image boundary are contour pixels; and

determining object contours based at least in part on determining contour pixels.

17. The system of claim 13 , wherein the memory further includes instructions that, when executed by the at least one processor, cause the system to:

project the object mesh structure from a viewpoint to a segmentation mask associated with the viewpoint; and

refine the segmentation mask based at least in part on projecting the object mesh structure.

18. The system of claim 13 , wherein the memory further includes instructions that, when executed by the at least one processor, cause the system to:

determine a ground plane intersecting the object mesh structure;

determine a portion of the object mesh structure below the ground plane; and

remove from the object mesh structure the portion of the object mesh structure below the ground plane.

19. The system of claim 13 , wherein the object mesh structure can be rotated to be viewed from each of the plurality of viewpoints, and wherein the object mesh structure is overlaid by the object image corresponding to the viewpoint from which the object mesh structure is viewed.

20. The system of claim 13 , wherein displaying the object mesh structure comprises:

capturing an actual client device environment through an image capture element of a client device; and

displaying the object mesh structure overlaying the actual client device environment.