Methods, storage media, and systems for selecting an optimal image frame within a capture window

1 . A method of selecting an optimal two-dimensional image frame within a capture window for a three-dimensional reconstruction pipeline, the method comprising:

detecting an actuation of a capture function at a capture device;

generating a capture window comprising a plurality of sequential image frames proximate to the detected actuation;

capturing, within the capture window, a plurality of candidate image frames comprising image data of a respective candidate image frame and sensor data of the capture device, wherein the plurality of candidate image frames are captured by a camera of the capture device at a capture frame rate, and wherein the sensor data of the capture device is associated with the plurality of candidate image frames and related to the capture frame rate;

selecting an optimal image frame of the plurality of candidate image frames in the capture window based on a lower frame cost relative to at least one other candidate image frame according to the sensor data of the capture device for each candidate image frame of the plurality of candidate image frames; and

storing the selected optimal image frame of the plurality of candidate image frames.

2 . The method of claim 1 , wherein the capture function is at least one of a physical button of the capture device and a digital button of the capture device.

3 . The method of claim 1 , wherein the capture window is spatially proximate to a pose associated with the detected actuation.

4 . The method of claim 3 , wherein a beginning of the capture window is greater than a first threshold translation from the pose associated with the detected actuation and an end of the capture window is less than a second threshold translation from the pose associated with the detected actuation.

5 . The method of claim 4 , wherein the first threshold translation and the second threshold translation are in units of pixels.

6 . The method of claim 4 , wherein the first threshold translation is 0 meters and the second threshold translation is 0.5 meters.

7 . The method of claim 3 , wherein a beginning of the capture window is greater than a first threshold rotation from the pose associated with the detected actuation and an end of the capture window is less than a second threshold rotation from the pose associated with the detected actuation.

8 . The method of claim 7 , wherein the first threshold rotation and the second threshold rotation are in units of pixels.

9 . The method of claim 7 , wherein the first threshold rotation is 0 radians and the second threshold rotation is 0.22 radians.

10 . The method of claim 3 , wherein capturing the plurality of candidate image frames is based on optical flow.

11 . The method of claim 1 , wherein the capture window is temporally proximate to a time associated with the detected actuation.

12 . The method of claim 11 , wherein a beginning of the capture window is greater than a first threshold time from the time associated with the detected actuation and an end of the capture window is less than a second threshold time from the time associated with the detected actuation.

13 . The method of claim 12 , wherein the first threshold time is 0 seconds and the second threshold time is 1 second.

14 . The method of claim 1 , wherein the sensor data comprises at least one of inertial sensor data, accelerometer sensor data, gyroscope sensor data, magnetometer sensor data, and altimeter sensor data.

15 . The method of claim 1 , wherein the sensor data comprises light sensor data, camera sensor data, visual data, and depth data.

16 . The method of claim 1 , wherein selecting the optimal image frame of the plurality of candidate image frames comprises selecting a least blurry image frame of the plurality of candidate image frames.

17 . The method of claim 1 , wherein selecting the optimal image frame of the plurality of candidate image frames comprises comparing sensor data associated with each candidate image frame of the plurality of candidate image frames to sensor data associated with each other candidate image frame of the plurality of candidate image frames.

18 . The method of claim 17 , wherein selecting the optimal image frame of the plurality of image frames comprises selecting an image frame of the plurality of candidate image frames associated with at least one of: a fastest shutter speed, a lowest degree of capture device rotation, a lowest rate of capture device rotation, a lowest degree of capture device translation, a lowest rate of capture device translation, fewest total number of blurred pixels, most architectural features, and most intersections of architectural features.

19 . A method of selecting an optimal two-dimensional image frame within a capture window for a three-dimensional reconstruction pipeline, the method comprising:

detecting an actuation of a capture function at a capture device;

generating a capture window comprising a plurality of image frames, wherein the capture window is spatially proximate to a pose associated with the detected actuation, and wherein a beginning of the capture window is greater than a first threshold translation from the pose associated with the detected actuation and an end of the capture window is less than a second threshold translation from the pose associated with the detected actuation;

capturing, within the capture window, a plurality of candidate image frames and sensor data of the capture device, wherein the plurality of image frames are captured by a camera of the capture device at a capture frame rate, and wherein the sensor data is associated with the plurality of candidate image frames and related to the capture frame rate;

selecting an optimal image frame of the plurality of candidate image frames based on at least one of image data of the plurality of candidate image frames and the sensor data of the capture device; and

storing the selected optimal image frame of the plurality of candidate image frames.

20 . A method of selecting an optimal two-dimensional image frame within a capture window for a three-dimensional reconstruction pipeline, the method comprising:

detecting an actuation of a capture function at a capture device;

generating a capture window comprising a plurality of image frames, wherein the capture window is spatially proximate to a pose associated with the detected actuation, and wherein a beginning of the capture window is greater than a first threshold rotation from the pose associated with the detected actuation and an end of the capture window is less than a second threshold rotation from the pose associated with the detected actuation;

capturing, within the capture window, a plurality of candidate image frames and sensor data of the capture device, wherein the plurality of image frames are captured by a camera of the capture device at a capture frame rate, and wherein the sensor data is associated with the plurality of candidate image frames and related to the capture frame rate;

selecting an optimal image frame of the plurality of candidate image frames based on at least one of image data of the plurality of candidate image frames and the sensor data of the capture device; and

storing the selected optimal image frame of the plurality of candidate image frames.