Granted Patent B2

US 10,382,673 · App. 15/404,044 · Granted Aug 13, 2019

Inventors: Manish Upendran (San Francisco, CA); William Castillo (San Carlos, CA); Derek Halliday (Oakland, CA)

Assignee: Hover Inc.

H04N5/23222G06T7/0002G06T17/00H04N5/2351H04N5/23206H04N5/23293G06T2200/08G06T2200/24G06T2200/28G06T2207/30168G06T2207/30184

Abstract

Claims (90)

1. A method for generating automatic prompts to improve quality of images used to create 3D models, comprising:

executing, on a device, an application for capturing images to be used to create 3D models;

wherein execution of the application causes the device to perform:

while a user is preparing to use the device to capture an image to be used to create a 3D model of a target object:

continuously displaying, on a display, visual content that would be captured if a capture control for the device were to be activated; and

causing one or more tests to be performed to determine whether a quality-based condition is satisfied, wherein the quality-based condition indicates whether the image is suitable for use in creating a 3D model of the target object;

while the quality-based condition is not satisfied, displaying on the display concurrent with the visual content that would be captured, a first visual indication that the quality-based condition is not satisfied;

while the quality-based condition is satisfied, ceasing to display, on the display of the device, the first visual indication;

in response to activation of the capture control, capturing an image.

2. The method of claim 1 further comprising exporting the image external from the device to enable the image to be used to create a 3D model of the target object.

3. The method of claim 1 further comprising:

while the quality-based condition is satisfied, displaying, on the display of the device, a second visual indication that the quality-based condition is satisfied.

4. The method of claim 1 further comprising:

after the image is captured, generating one or more visual prompts on the display for the user to specify identifiers for one or more objects depicted in the image; and

exporting, in association with the image, a set of metadata that establishes a correlation between the identifiers and coordinates in the image.

5. The method of claim 4 wherein generating one or more visual prompts comprises:

analyzing the image to determine one or more candidate identifiers;

wherein the one or more visual prompts include an indication of the one or more candidate identifiers.

6. The method of claim 1 wherein causing one or more tests to be performed to determine whether a quality-based condition is satisfied comprises:

monitoring one or more sensors of the device; and

repeatedly performing comparisons between output generated by the one or more sensors and a threshold to determine whether the quality-based condition is satisfied.

7. The method of claim 6 wherein the one or more sensors include at least an accelerometer.

8. The method of claim 6 wherein the one or more sensors include at least an altimeter.

9. The method of claim 6 wherein the one or more sensors include at least a gyroscope.

10. The method of claim 6 wherein the one or more sensors include at least a light sensor.

11. The method of claim 6 wherein the one or more sensors include at least a proximity sensor.

12. The method of claim 1 wherein causing the one or more tests to be performed to determine whether a quality-based condition is satisfied comprises:

analyzing the visual content that would be captured to determine whether the quality-based condition is satisfied.

13. The method of claim 12 further comprising:

capturing a plurality of images; and

analyzing each image in the plurality of images.

14. The method of claim 1 further comprising:

causing one or more additional tests to be performed to determine whether the image satisfies a second quality-based condition;

if the image does not satisfy the second quality-based condition, displaying on the display an indication that the second quality-based condition is not met.

15. The method of claim 1 wherein the first visual indication specifies the quality-based condition.

16. A method comprising:

executing, on a device, an application for capturing images to be used to create 3D models;

wherein execution of the application causes the device to perform:

while a user is preparing to use the device to capture an image to be used to create a 3D model of a target object:

continuously displaying, on a display, visual content that would be captured if a capture control for the device were to be activated;

transmitting the visual content to a computing device external to the device; and

receiving, from the computing device, an indication of whether a quality-based condition is satisfied, wherein the quality-based condition indicates whether the image is suitable for use in creating a 3D model of the target object;

while the quality-based condition is not satisfied, displaying on the display concurrent with the visual content that would be captured, a first visual indication that the quality-based condition is not satisfied;

while the quality-based condition is satisfied, ceasing to display, on the display of the device the first visual indication;

in response to activation of the capture control, capturing an image.

17. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause performance of:

executing, on a device, an application for capturing images to be used to create 3D models;

wherein execution of the application causes the device to perform:

while a user is preparing to use the device to capture an image to be used to create a 3D model of a target object:

continuously displaying, on a display, visual content that would be captured if a capture control for the device were to be activated; and

causing one or more tests to be formed to determine whether a quality-based condition is satisfied, wherein the quality-based condition indicates whether the image is suitable for use in creating a 3D model of the target object;

while the quality-based condition is not satisfied, displaying on the display concurrent with the visual content that would be captured, a first visual indication that the quality-based condition is not satisfied;

while the quality-based condition is satisfied, ceasing to display, on the display of the device, the first visual indication;

in response to activation of the capture control, capturing an image.

18. The one or more non-transitory computer-readable media of claim 17 further comprising instructions for exporting the image external from the device to enable the image to be used to create a 3D model of the target object.

19. The one or more non-transitory computer-readable media of claim 17 further comprising instructions for:

while the quality-based condition is satisfied, displaying, on the display of the device, a second visual indication that the quality-based condition is satisfied.

20. The one or more non-transitory computer-readable media of claim 17 further comprising instructions for:

after the image is captured, generating one or more visual prompts on the display for the user to specify identifiers for one or more objects depicted in the image; and

exporting, in association with the image, a set of metadata that establishes a correlation between the identifiers and coordinates in the image.

21. The one or more non-transitory computer-readable media of claim 20 wherein generating one or more visual prompts comprises:

analyzing the image to determine one or more candidate identifiers;

wherein the one or more visual prompts include an indication of the one or more candidate identifiers.

22. The one or more non-transitory computer-readable media of claim 17 wherein causing one or more tests to be performed to determine whether a quality-based condition is satisfied comprises:

monitoring one or more sensors of the device; and

repeatedly performing comparisons between output generated by the one or more sensors and a threshold to determine whether the quality-based condition is satisfied.

23. The one or more non-transitory computer-readable media of claim 22 wherein the one or more sensors include at least an accelerometer.

24. The one or more non-transitory computer-readable media of claim 22 wherein the one or more sensors include at least an altimeter.

25. The one or more non-transitory computer-readable media of claim 22 wherein the one or more sensors include at least a gyroscope.

26. The one or more non-transitory computer-readable media of claim 22 wherein the one or more sensors include at least a light sensor.

27. The one or more non-transitory computer-readable media of claim 22 wherein the one or more sensors include at least a proximity sensor.

28. The one or more non-transitory computer-readable media of claim 17 wherein causing the one or more tests to be performed to determine whether a quality-based condition is satisfied comprises:

analyzing the visual content that would be captured to determine whether the quality-based condition is satisfied.

29. The one or more non-transitory computer-readable media of claim 28 further comprising instructions for:

capturing a plurality of images; and

analyzing each image in the plurality of images.

30. The one or more non-transitory computer-readable media of claim 17 further comprising instructions for:

causing one or more additional tests to be performed to determine whether the image satisfies a second quality-based condition;

if the image does not satisfy the second quality-based condition, displaying on the display an indication that the second quality-based condition is not met.

31. The one or more non-transitory computer-readable media of claim 17 wherein the first visual indication specifies the quality-based condition.

32. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause performance of:

executing, on a device, an application for capturing images to be used to create 3D models;

wherein execution of the application causes the device to perform:

while a user is preparing to use the device to capture an image to be used to create a 3D model of a target object:

continuously displaying, on a display, visual content that would be captured if a capture control for the device were to be activated;

transmitting the visual content to a computing device external to the device; and

receiving, from the computing device, an indication of whether a quality-based condition is satisfied, wherein the quality-based condition indicates whether the image is suitable for use in creating a 3D model of the target object;

while the quality-based condition is not satisfied, displaying on the display concurrent with the visual content that would be captured, a first visual indication that the quality-based condition is not satisfied;

while the quality-based condition is satisfied, ceasing to display, on the display of the device, the first visual indication;

in response to activation of the capture control, capturing an image.

Assignments (9)

Continuity (1)

Cited By (4)

US 12,316,957 US 12,445,717 US 12,684,230 US 12,695,982