Plate reconstruction of obscured views of captured imagery using arbitrary captured inputs
An imagery processing system obtains capture inputs from capture devices that might have capture parameters and characteristics that differ from those of a main imagery capture device. By normalizing outputs of those capture devices, potentially arbitrary capture devices could be used for reconstructing portions of a scene captured by the main imagery capture device when reconstructing a plate of the scene to replace an object in the scene with what the object obscured in the scene. Reconstruction could be of one main image, a stereo pair of images, or some number, N, of images where N>2.
1. A computer-implemented method of reconstructing a plate by processing images of a scene from a main imaging device using capture device inputs from capture devices recording the same scene, the method comprising:
receiving reference stage parameters and normalization parameters associated with the capture devices;
storing the normalization parameters;
receiving main input from the main imaging device, including a primary object that is at least partially obscuring an obscured object;
receiving the capture device inputs from the capture devices;
normalizing the capture device inputs using normalization parameters to form normalized captured inputs;
identifying a plate for reconstruction that includes the primary object;
aligning two or more normalized captured inputs;
obtaining reconstruction selections in the identified plate to remove the primary object from the plate; and
reconstructing the plate based on the selections, the normalized aligned captured inputs, and the reconstruction selections to form a reconstructed plate showing the portions of the obscured object that were obscured by the primary object.
2. The method of claim 1 , wherein the main imaging device includes a camera, the method further comprising:
replacing pixels in digitized video content captured with the camera to modify a captured scene in the digitized video content, wherein pixel color values of replaced pixels are determined based, at least in part, on the reconstruction selections and the capture device inputs.
3. The method of claim 2 , further comprising:
determining objects to be removed from the captured scene; and
using results of determining for a plate reconstruction.
4. The method of claim 1 , wherein the reference stage parameters relate at least in part to what capture devices are being used and differences between the capture devices and the main imaging device.
5. The method of claim 1 , the capture devices comprising:
one or more of a first camera having a resolution different than the main imaging device and/or a second camera optimized for a spectrum different than that of the main imaging device.
6. The method of claim 1 , further comprising:
configuring the capture devices based on the normalization parameters.
7. An apparatus for reconstructing a plate by processing images of a scene from a main imaging device using capture device inputs from capture devices recording the same scene, the apparatus comprising:
one or more processors;
one or more non-transitory processor-readable media including instructions executable by the one or more processors for:
receiving reference stage parameters and normalization parameters associated with the capture devices;
storing the normalization parameters;
receiving main input from the main imaging device, including a primary object that is at least partially obscuring an obscured object;
receiving the capture device inputs from the capture devices;
normalizing the capture device inputs using normalization parameters to form normalized captured inputs;
identifying a plate for reconstruction that includes the primary object;
aligning two or more normalized captured inputs;
obtaining reconstruction selections in the identified plate to remove the primary object from the plate; and
reconstructing the plate based on the selections, the normalized aligned captured inputs, and the reconstruction selections to form a reconstructed plate showing the portions of the obscured object that were obscured by the primary object.
8. The apparatus of claim 7 , wherein the main imaging device includes a camera, the apparatus further comprising:
replacing pixels in digitized video content captured with the camera to modify a captured scene in the digitized video content, wherein pixel color values of replaced pixels are determined based, at least in part, on the reconstruction selections and the capture device inputs.
9. The apparatus of claim 8 , further comprising:
determining objects to be removed from the captured scene; and
using results of determining for a plate reconstruction.
10. The apparatus of claim 7 , wherein the reference stage parameters relate at least in part to what capture devices are being used and differences between the capture devices and the main imaging device.
11. The apparatus of claim 7 , the capture devices comprising:
one or more of a first camera having a resolution different than the main imaging device and/or a second camera optimized for a spectrum different than that of the main imaging device.
12. The apparatus of claim 7 , further comprising:
configuring the capture devices based on the normalization parameters.
13. A non-transitory processor-readable medium for reconstructing a plate by processing images of a scene from a main imaging device using capture device inputs from capture devices recording the same scene, the non-transitory processor-readable medium including instructions executable by one or more processors for:
receiving reference stage parameters and normalization parameters associated with the capture devices;
storing the normalization parameters;
receiving main input from the main imaging device, including a primary object that is at least partially obscuring an obscured object;
receiving the capture device inputs from the capture devices;
normalizing the capture device inputs using normalization parameters to form normalized captured inputs;
identifying a plate for reconstruction that includes the primary object;
aligning two or more normalized captured inputs;
obtaining reconstruction selections in the identified plate to remove the primary object from the plate; and
reconstructing the plate based on the selections, the normalized aligned captured inputs, and the reconstruction selections to form a reconstructed plate showing the portions of the obscured object that were obscured by the primary object.
14. The non-transitory processor-readable medium of claim 13 , wherein the main imaging device includes a camera, further comprising:
replacing pixels in digitized video content captured with the camera to modify a captured scene in the digitized video content, wherein pixel color values of replaced pixels are determined based, at least in part, on the reconstruction selections and the capture device inputs.
15. The non-transitory processor-readable medium of claim 14 , further comprising:
determining objects to be removed from the captured scene; and
using results of determining for a plate reconstruction.
16. The non-transitory processor-readable medium of claim 13 , wherein the reference stage parameters relate at least in part to what capture devices are being used and differences between the capture devices and the main imaging device.
17. The non-transitory processor-readable medium of claim 13 , the capture devices comprising:
one or more of a first camera having a resolution different than the main imaging device and/or a second camera optimized for a spectrum different than that of the main imaging device.
18. The non-transitory processor-readable medium of claim 13 , further comprising:
configuring the capture devices based on the normalization parameters.
19. The method of claim 1 , wherein a normalization parameter includes a resolution of a particular capture device.
20. The method of claim 19 , further comprising:
upscaling the resolution of the at least one of the capture devices to cause images captured by the particular capture device to be closer to a resolution of the main imaging device.