Method for separating local and remote content in a camera-projector based collaborative system

1. An image processing method for collaborating between a first augmented reality (AR) workspace and a second AR workspace, the method comprising:

calibrating the first AR workspace by creating:

a camera-to-shared space transformation between a camera coordinate space and a shared coordinate space; and

a shared space-to-projector transformation between the shared coordinate space and a projector coordinate space;

obtaining a remote composite image of a foreground content in the second AR workspace, wherein the remote composite image is formatted in the shared coordinate space;

generating a projector image by applying the shared space-to-projector transformation to the remote composite image;

projecting the projector image into the first AR workspace;

obtaining an input image of the first AR workspace that includes the projector image and a foreground content of the first AR workspace, wherein the input image is formatted in the camera coordinate space;

generating an output image by applying the camera-to-shared space transformation to the input image, wherein the output image is formatted in the shared coordinate space;

obtaining a remote mask based on the remote composite image;

eliminating the projector image and preserving the foreground content of the first AR workspace by applying the remote mask to the output image and generating a masked output image; and

sending the masked output image to the second AR workspace.

2. The method of claim 1 , wherein

calibrating the first AR workspace further comprises creating:

a camera-to-projector transformation between the camera coordinate space and the projector coordinate space by projecting a target image into the first AR workspace and then imaging the target image;

a projector-to-world transformation between the projector coordinate space and a physical coordinate space of the first AR workspace by imaging a physical target of known dimensions in the first AR workspace; and

a camera-to-world transformation by combining the camera-to-projector transformation and the projector-to-world transformation.

3. The method of claim 1 , wherein applying the camera-to-shared space transformation to the input image further includes applying a normalization scheme that generates the output image with standard dimensions and a standard pixel resolution in the shared coordinate space.

4. The method of claim 3 , wherein the standard dimensions are proportional to an area of a working region that is captured in the input image of the first AR workspace.

5. The method of claim 4 , further comprising:

after applying the normalization scheme, enhancing contrast between background pixels of the output image that correspond to a surface the working region and foreground pixels of the output image that correspond to the foreground content in the first AR workspace.

6. The method of claim 5 , wherein

the working region is a piece of paper disposed in the first AR workspace,

the background pixels correspond to a color of the paper,

the foreground content comprises a marking on the paper, and

the foreground pixels correspond to a color and an opacity of the marking.

7. The method of claim 3 , wherein

the second AR workspace includes a plurality of remote workstations that each output a remote image formatted with the standard dimensions and the standard resolution of the shared coordinate space, and

obtaining the remote composite image includes obtaining the plurality of remote images and merging the plurality of remote images into the remote composite image.

8. The method of claim 7 , wherein

merging the plurality of remote images includes applying a minimum operator to the plurality of remote images and determining a merged pixel value for each pixel in the shared coordinate space, and

for each given pixel in the shared coordinate space, the minimum operator determines the merged pixel value by selecting the remote image with a highest contrast foreground pixel value, relative to a background pixel value in the same remote image, at the given pixel.

9. The method of claim 1 , further comprising:

generating a collaboration image by merging the masked output image and the remote composite image, wherein

the collaboration image is formatted in the shared coordinate space.

10. The method of claim 1 , wherein

the input image of the first AR workspace is obtained with a Projection with Interactive Capture (PIC) device that comprises a camera and a projector, and

the projector image is projected into the first AR workspace by the projector.

11. A non-transitory computer readable medium (CRM) storing computer readable program code for collaborating between a first augmented reality (AR) workspace and a second AR workspace, the computer readable program code causes a computer to:

calibrate the first AR workspace by creating:

a camera-to-shared space transformation between a camera coordinate space and a shared coordinate space; and

a shared space-to-projector transformation between the shared coordinate space and a projector coordinate space;

obtain a remote composite image of a foreground content in the second AR workspace, wherein the remote composite image is formatted in the shared coordinate space;

generate a projector image by applying the shared space-to-projector transformation to the remote composite image;

project the projector image into the first AR workspace;

obtain an input image of the first AR workspace that includes the projector image and a foreground content of the first AR workspace, wherein the input image is formatted in the camera coordinate space;

generate an output image by applying the camera-to-shared space transformation to the input image, wherein the output image is formatted in the shared coordinate space;

obtain a remote mask based on the remote composite image;

eliminate the projector image and preserve the foreground content of the first AR workspace by applying the remote mask to the output image and generating a masked output image; and

send the masked output image to the second AR workspace.

12. The CRM of claim 11 , wherein, in calibrating the first AR workspace, the computer readable program code further causes the computer to create:

a camera-to-projector transformation between the camera coordinate space and the projector coordinate space by projecting a target image into the first AR workspace and then imaging the target image;

a projector-to-world transformation between the projector coordinate space and a physical coordinate space of the first AR workspace by imaging a physical target of known dimensions in the first AR workspace; and

a camera-to-world transformation by combining the camera-to-projector transformation and the projector-to-world transformation.

13. The CRM of claim 11 , wherein, in applying the camera-to-shared space transformation to the input image, the computer readable program code further causes the computer to further apply a normalization scheme that generates the output image with standard dimensions and a standard pixel resolution in the shared coordinate space.

14. The CRM of claim 11 , wherein the computer readable program code further causes the computer to:

generate a collaboration image by merging the masked output image and the remote composite image, wherein

the collaboration image is formatted in the shared coordinate space.

15. The CRM of claim 11 , wherein computer readable program code further causes the computer to:

obtain the input image of the first AR workspace with a Projection with Interactive Capture (PIC) device that comprises a camera and a projector, and

project the projector image into the first AR workspace with the projector of the PIC device.

16. A system for collaborating between a first augmented reality (AR) workspace and a second AR workspace, the system comprising:

a memory; and

a processor coupled to the memory, wherein the processor:

calibrates the first AR workspace by creating:

a camera-to-shared space transformation between a camera coordinate space and a shared coordinate space; and

a shared space-to-projector transformation between the shared coordinate space and a projector coordinate space;

obtains a remote composite image of a foreground content in the second AR workspace, wherein the remote composite image is formatted in the shared coordinate space;

generates a projector image by applying the shared space-to-projector transformation to the remote composite image;

projects the projector image into the first AR workspace;

obtains an input image of the first AR workspace that includes the projector image and a foreground content of the first AR workspace, wherein the input image is formatted in the camera coordinate space;

generates an output image by applying the camera-to-shared space transformation to the input image, wherein the output image is formatted in the shared coordinate space;

obtains a remote mask based on the remote composite image;

eliminates the projector image and preserves the foreground content of the first AR workspace by applying the remote mask to the output image and generating a masked output image; and

sends the masked output image to the second AR workspace.

17. The system of claim 16 , wherein, in calibrating the first AR workspace, the processor further creates:

a camera-to-projector transformation between the camera coordinate space and the projector coordinate space by projecting a target image into the first AR workspace and then imaging the target image;

a projector-to-world transformation between the projector coordinate space and a physical coordinate space of the first AR workspace by imaging a physical target of known dimensions in the first AR workspace; and

a camera-to-world transformation by combining the camera-to-projector transformation and the projector-to-world transformation.

18. The system of claim 16 , wherein, in applying the camera-to-shared space transformation to the input image, the processor further applies a normalization scheme that generates the output image with standard dimensions and a standard pixel resolution in the shared coordinate space.

19. The system of claim 16 , wherein the processor further:

generates a collaboration image by merging the masked output image and the remote composite image, wherein

the collaboration image is formatted in the shared coordinate space.

20. The system of claim 16 , wherein

the input image of the first AR workspace is obtained with a Projection with Interactive Capture (PIC) device that comprises a camera and a projector, and

the projector image is projected into the first AR workspace by the projector.