Difference matting utilising videowall
An image processing method is carried out by a processor unit configured to input the first image from the camera to determine the portions to be masked in a first image comprising at least one object in front of a video wall display in which a displayed background image is shown and a captured background image of the displayed background image being captured by the camera. Accordingly, the method comprises the steps of determining a camera position and a camera point of view from which the camera receives the first image, accessing a background image that is the source of the displayed background image shown on the video wall display when the camera receives the first image, and creating an edited background image by editing the accessed background image according to a video wall display position information recorded in a memory unit.
1 . An image processing method carried out by a processor unit configured to input a first image from a camera to determine which unwanted background portions of the first image are to be masked, the method comprising:
receiving, from a position tracking device, a camera position and a camera point of view from which the camera acquired the first image, the first image including at least one object in front of a video wall display and including a captured background image that is the displayed background image as captured by the same camera,
accessing a background image that is the source of the displayed background image shown on the video wall display when the camera acquired the first image,
creating an edited background image by converting the accessed background image into a form viewed by the camera using positions and dimensions of the video wall display stored in a memory unit, and using together with the camera position and the camera point of view, and applying, to the accessed background image, optical deformations applied by the camera, to generate an edited background image that is the background image as viewed by the camera, and
determining the portions to be masked in the first image by performing difference keying between the first image and the edited background image to determine unwanted background portions in the first image.
2 . The image processing method according to claim 1 , further comprising:
determining the camera parameters of the camera with which the camera acquired the first image prior to the step of determining the portions to be masked, and
wherein the applying, to the accessed background image, optical deformations applied by the camera is performed according to the camera parameters.
3 . The image processing method according to claim 2 , wherein the camera parameters are at least one of lens zoom state, the filter used, lens focus state, lens type, or lens dimensions.
4 . The image processing method according to claim 1 , wherein the image processing method creates an edited background image according to the distance of each pixel in the accessed background image from the sensor of the camera when creating the edited background image.
5 . The image processing method according to claim 1 , wherein the background image comprises images in the form of a linear content obtained from a 3D virtual scene model.
6 . A system comprising:
a processor unit configured to input an image captured by a camera to determine which unwanted background portions of a first image are to be masked, the first image comprising at least one object in front of a video wall display in which a displayed background image is shown, wherein a position tracking device is configured to send a point of view and position information of the camera to the processor unit, and wherein the processor unit is configured to perform the following steps:
receiving, from the position tracking device, a camera position and a camera point of view from which the camera acquired the first image, the first image including a captured background image that is the displayed background image as captured by the same camera,
accessing a background image that is the source of the displayed background image shown on the video wall display when the camera acquired the first image,
creating an edited background image by converting the accessed background image into a form viewed by the camera using positions and dimensions of the video wall display stored in a memory unit, and using the camera position and the camera point of view, and applying, to the accessed background image, optical deformations applied by the camera, to generate an edited background image that is the background image as viewed by the camera, and
determining the portions to be masked in the first image by performing difference keying between the first image and the edited background image to determine unwanted background portions in the first image.
7 . The system according to claim 6 , wherein the processor unit is further configured to perform the following steps:
determining the camera parameters of the camera with which the camera acquired the first image,
wherein the applying, to the accessed background image, optical deformations applied by the camera is performed according to the camera parameters.
8 . The system according to claim 7 , wherein the camera parameters are at least one of lens zoom state, the filter used, lens focus state, lens type, or lens dimensions.
9 . The system according to claim 6 , wherein the processor unit is configured to create an edited background image according to the distance of each pixel in the accessed background image from the sensor of the camera when creating the edited background image.
10 . The system according to claim 6 , wherein the background image comprises images in the form of a linear content obtained from a 3D virtual scene model.