TECHNIQUES FOR ENHANCED IMAGE CAPTURE USING A COMPUTER-VISION NETWORK
Disclosed are techniques for enhancing two-dimensional (2D) image capture of subjects (e.g., a physical structure, such as a residential building) to maximize the feature correspondences available for three-dimensional (3D) model reconstruction. More specifically, disclosed is a computer-vision network configured to provide viewfinder interfaces and analyses to guide the improved capture of an intended subject for specified purposes. Additionally, the computer-vision network can be configured to generate a metric representing a quality of feature correspondences between images of a complete set of images used for reconstructing a 3D model of a physical structure. The computer-vision network can also be configured to generate feedback at or before image capture time to guide improvements to the quality of feature correspondences between a pair of images.
1 - 7 . (canceled)
8 . A computer-implemented method, comprising:
receiving a first set of pixels of a first image frame representing a scene visible to an image capturing device;
generating a first segmentation mask associated with a physical structure depicted within the first set of pixels of the first image frame, the first segmentation mask including one or more first segmentation pixels;
receiving a second set of pixels of a second image frame representing the scene visible to the image capturing device;
generating a second segmentation mask associated with the physical structure depicted within the second set of pixels of the second image frame, the second segmentation mask including one or more second segmentation pixels, and at least one first segmentation pixel being different from at least one second segmentation pixel; and
generating an accumulated segmentation mask based on aggregating one or more pixels of the first segmentation mask and the second segmentation mask.
9 . The computer-implemented method of claim 8 , wherein aggregating one or more pixels of the first segmentation mask and the second segmentation mask comprises pixel voting the one or more pixels of the first and second segmentation masks.
10 . The computer-implemented method of claim 9 , wherein pixel voting comprises applying a pixel vote value to the one or more pixels of the first and second segmentation masks.
11 . The computer-implemented method of claim 10 , wherein pixel voting comprises weighting one or more pixel votes of the one or more pixels of the second segmentation mask higher than one or more pixel votes of the one or more pixels of the first segmentation mask.
12 . The computer-implemented method of claim 10 , wherein the accumulated segmentation mask comprises aggregated pixel values exceeding a threshold pixel vote value.
13 . The computer-implemented method of claim 8 , further comprising applying a feedback signal indicating the physical structure's position in a display coupled to the image capturing device.
14 . The computer-implemented method of claim 13 , wherein the feedback signal is a bounding box enveloping the accumulated segmentation mask.
15 . The computer-implemented method of claim 14 , further comprising projecting the bounding box into a third image frame representing the scene visible to the image capturing device.
16 . A system comprising:
one or more processors; and
one or more memory devices storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving a first set of pixels of a first image frame representing a scene visible to an image capturing device;
generating a first segmentation mask associated with a physical structure depicted within the first set of pixels of the first image frame, the first segmentation mask including one or more first segmentation pixels;
receiving a second set of pixels of a second image frame representing the scene visible to the image capturing device;
generating a second segmentation mask associated with the physical structure depicted within the second set of pixels of the second image frame, the second segmentation mask including one or more second segmentation pixels, and at least one first segmentation pixel being different from at least one second segmentation pixel; and
generating an accumulated segmentation mask based on aggregating one or more pixels of the first segmentation mask and the second segmentation mask.
17 . The system of claim 16 , wherein aggregating one or more pixels of the first segmentation mask and the second segmentation mask comprises pixel voting the one or more pixels of the first and second segmentation masks.
18 . The system of claim 17 , wherein pixel voting comprises applying a pixel vote value to the one or more pixels of the first and second segmentation masks.
19 . The system of claim 18 , wherein pixel voting comprises weighting one or more pixel votes of the one or more pixels of the second segmentation mask higher than one or more pixel votes of the one or more pixels of the first segmentation mask.
20 . The system of claim 18 , wherein the accumulated segmentation mask comprises aggregated pixel values exceeding a threshold pixel vote value.
21 . The system of claim 16 , further comprising applying a feedback signal indicating the physical structure's position in a display coupled to the image capturing device.
22 . The system of claim 21 , wherein the feedback signal is a bounding box enveloping the accumulated segmentation mask.
23 . The system of claim 22 , further comprising projecting the bounding box into a third image frame representing the scene visible to the image capturing device.