Granted Patent B1

US 9,618,940 · App. 15/068,292 · Granted Apr 11, 2017

Abstract

Claims (104)

1. A computerized method performed by an unmanned aerial vehicle (UAV) system comprising one or more processors, the method comprising:

in initial steps:

receiving, by an unmanned aerial vehicle (UAV), flight information describing a job to perform an inspection of a rooftop, the flight information specifying a flight pattern according to which the UAV is to navigate over the rooftop during an inspection and is to make turns to change direction of flight of the UAV;

navigating, by the UAV, to at least a particular altitude;

in subsequent steps:

performing the inspection of the rooftop according to the flight pattern specified in the received flight information, the inspection including obtaining sensor information describing the rooftop;

determining, by the one or more processors executing a visual classifier trained to identify rooftop damage, that a particular area of the rooftop identified from the obtained sensor information is indicated as being damaged;

determining, by the one or more processors, a corresponding location associated with the particular area of the rooftop indicated as being damaged, and navigating the UAV to a position above the corresponding location associated with the particular area of the rooftop;

performing, by the UAV, a detailed inspection of the particular area of the rooftop indicated as being damaged, including obtaining detailed sensor information describing the particular area of the rooftop; and

navigating the UAV to a safe landing location.

2. The method of claim 1 , wherein the particular altitude indicates an altitude that is free of obstructions within a property boundary indicated in the information describing the job.

3. The method of claim 1 , wherein the UAV enforces a geofence envelope limiting the UAV to real-world locations within a property boundary indicated in the flight information describing the job.

4. The method of claim 1 , wherein performing an inspection of the particular area comprises:

descending from the position above the corresponding location, and while descending, obtaining sensor information of the particular area;

determining that the UAV is within a threshold distance of the rooftop; and

ascending along a vertical direction from the corresponding location, and while ascending, obtaining sensor data of the particular area.

5. The method of claim 4 , wherein the sensor information obtained during the inspection of the rooftop includes images of the rooftop, and wherein the method further comprises:

obtaining one or more images, obtained during the inspection of the rooftop, that include the particular area,

wherein descending comprises:

performing visual odometry using the obtained images and sensor information obtained as the UAV descends, and

correcting a flight path of the UAV, along a horizontal direction, while descending based on the performed visual odometry.

6. The method of claim 1 , further comprising:

subsequent to traveling to the safe landing location, generating, by the UAV, a report for presentation on a user device of an operator, wherein the report includes a graphical representation of the rooftop and a location within the graphical representation of the damaged area, and wherein the report comprises an interactive document or a static document.

7. The method of claim 6 , further comprising:

determining, using the visual classifier, one or more features indicative of rooftop damage that are included in detailed sensor information describing the damaged area; and

including, in the report, information describing the one or more features.

8. The method of claim 1 , wherein the visual classifier determines rooftop damage based on one or more features included in sensor information that are indicative of rooftop damage, wherein a feature comprises: cracks in ceramic/clay tiles, broken corners of tiles, gaps between shingles, curled edges of a rooftop, dents in a wooden rooftop, dents or divots in tiles, moss or algae on shingles, shingle splitting, surface nail damage, spatter marks, or nails raised above a surface of the rooftop.

9. The method of claim 1 , wherein determining that a particular area of a rooftop is indicated as being damaged comprises:

obtaining sensor information describing the rooftop at a prior point in time;

comparing the sensor information associated with the prior point in time with the sensor information describing the rooftop obtained during the inspection of the rooftop; and

determining, using the visual classifier, that the particular area of the rooftop is indicated as being damaged based on the comparisons.

10. A non-transitory computer storage medium comprising instructions that when executed by one or more processors included in an Unmanned Aerial Vehicle (UAV), cause the UAV to perform operations comprising:

in initial steps:

receiving, by the UAV, flight information describing a job to perform an inspection of a rooftop, the flight information specifying a flight pattern according to which the UAV is to navigate over the rooftop during an inspection and is to make turns to change direction of flight of the UAV;

navigating, by the UAV, to at least a particular altitude;

in subsequent steps:

performing the inspection of the rooftop according to the flight pattern specified in the received flight information, the inspection including obtaining sensor information describing the rooftop;

determining, by the UAV executing a visual classifier trained to identify rooftop damage, that a particular area of the rooftop identified from the obtained sensor information is indicated as being damaged;

determining, by the UAV, a corresponding location associated with the particular area of the rooftop indicated as being damaged, and navigating the UAV to a position above the corresponding location associated with the particular area of the rooftop;

performing, by the UAV, a detailed inspection of the particular area of the rooftop indicated as being damaged, including obtaining detailed sensor information describing the particular area of the rooftop; and

navigating the UAV to a safe landing location.

11. The computer storage medium of claim 10 , wherein the particular altitude indicates an altitude that is free of obstructions within a property boundary indicated in the information describing the job.

12. The computer storage medium of claim 10 , wherein the UAV enforces a geofence envelope limiting the UAV to real-world locations within a property boundary indicated in the flight information describing the job.

13. The computer storage medium of claim 10 , wherein performing an inspection of the particular area comprises:

descending from the position above the corresponding location, and while descending, obtaining sensor information of the particular area;

determining that the UAV is within a threshold distance of the rooftop; and

ascending along a vertical direction from the corresponding location, and while ascending, obtaining sensor data of the particular area.

14. The computer storage medium of claim 13 , wherein the sensor information obtained during the inspection of the rooftop includes images of the rooftop, and wherein the operations further comprise:

obtaining one or more images, obtained during the inspection of the rooftop, that include the particular area,

wherein descending comprises:

performing visual odometry using the obtained images and sensor information obtained as the UAV descends, and

correcting a flight path of the UAV, along a horizontal direction, while descending based on the performed visual odometry.

15. The computer storage medium of claim 10 , wherein the operations further comprise:

subsequent to traveling to the safe landing location, generating, by the UAV, a report for presentation on a user device of an operator, wherein the report includes a graphical representation of the rooftop and a location within the graphical representation of the damaged area, and wherein the report comprises an interactive document or a static document.

16. The computer storage medium of claim 15 , wherein the operations further comprise:

determining, using the visual classifier, one or more features indicative of rooftop damage that are included in detailed sensor information describing the damaged area; and

including, in the report, information describing the one or more features.

17. The computer storage medium of claim 10 , wherein the visual classifier determines rooftop damage based on one or more features included in sensor information that are indicative of rooftop damage, wherein a feature comprises: cracks in ceramic/clay tiles, broken corners of tiles, gaps between shingles, curled edges of a rooftop, dents in a wooden rooftop, dents or divots in tiles, moss or algae on shingles, shingle splitting, surface nail damage, spatter marks, or nails raised above a surface of the rooftop.

18. The computer storage medium of claim 10 , wherein determining that a particular area of a rooftop is indicated as being damaged comprises:

obtaining sensor information describing the rooftop at a prior point in time;

comparing the sensor information associated with the prior point in time with the sensor information describing the rooftop obtained during the inspection of the rooftop; and

determining, using the visual classifier, that the particular area of the rooftop is indicated as being damaged based on the comparisons.

19. A system included in an unmanned aerial vehicle (UAV) comprising one or more processors, and a non-transitory computer storage media storing instructions, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:

in initial steps:

receiving flight information describing a job to perform an inspection of a rooftop, the flight information specifying a flight pattern according to which the UAV is to navigate over the rooftop during an inspection and is to make turns to change direction of flight of the UAV;

navigating, by the UAV, to at least a particular altitude;

in subsequent steps:

performing, by the UAV, the inspection of the rooftop according to the flight pattern specified in the received flight information, the inspection including obtaining sensor information describing the rooftop;

determining, using an executed visual classifier trained to identify rooftop damage, that a particular area of the rooftop identified from the obtained sensor information is indicated as being damaged;

determining a corresponding location associated with the particular area of the rooftop indicated as being damaged, and navigating the UAV to a position above the corresponding location associated with the particular area of the rooftop;

performing, by the UAV, a detailed inspection of the particular area of the rooftop indicated as being damaged, including obtaining detailed sensor information describing the particular area of the rooftop; and

navigating, by the UAV, to a safe landing location.

20. The system of claim 19 , wherein the particular altitude indicates an altitude that is free of obstructions within a property boundary indicated in the information describing the job.

21. The system of claim 19 , wherein the UAV enforces a geofence envelope limiting the UAV to real-world locations within a property boundary indicated in the flight information describing the job.

22. The system of claim 19 , wherein performing an inspection of the particular area comprises:

descending, by the UAV, from the position above the corresponding location, and while descending, obtaining sensor information of the particular area;

determining that the UAV is within a threshold distance of the rooftop; and

ascending, by the UAV, along a vertical direction from the corresponding location, and while ascending, obtaining sensor data of the particular area.

23. The system of claim 22 , wherein the sensor information obtained during the inspection of the rooftop includes images of the rooftop, and wherein the operations further comprise:

obtaining one or more images, obtained during the inspection of the rooftop, that include the particular area,

wherein descending comprises:

performing visual odometry using the obtained images and sensor information obtained as the UAV descends, and

correcting a flight path of the UAV, along a horizontal direction, while descending based on the performed visual odometry.

24. The system of claim 19 , wherein the operations further comprise:

subsequent to traveling to the safe landing location, generating, by the UAV, a report for presentation on a user device of an operator, wherein the report includes a graphical representation of the rooftop and a location within the graphical representation of the damaged area, and wherein the report comprises an interactive document or a static document.

25. The system of claim 24 , wherein the operations further comprise:

determining, using the visual classifier, one or more features indicative of rooftop damage that are included in detailed sensor information describing the damaged area; and

including, in the report, information describing the one or more features.

26. The system of claim 19 , wherein the visual classifier determines rooftop damage based on one or more features included in sensor information that are indicative of rooftop damage, wherein a feature comprises: cracks in ceramic/clay tiles, broken corners of tiles, gaps between shingles, curled edges of a rooftop, dents in a wooden rooftop, dents or divots in tiles, moss or algae on shingles, shingle splitting, surface nail damage, spatter marks, or nails raised above a surface of the rooftop.

27. The system of claim 19 , wherein determining that a particular area of a rooftop is indicated as being damaged comprises:

obtaining sensor information describing the rooftop at a prior point in time;

comparing the sensor information associated with the prior point in time with the sensor information describing the rooftop obtained during the inspection of the rooftop; and

determining, using the visual classifier, that the particular area of the rooftop is indicated as being damaged based on the comparisons.

28. A method performed by an unmanned aerial vehicle (UAV) comprising one or more processors and one or more cameras, the method comprising:

obtaining, by the UAV, first sensor information describing a particular area of a rooftop, the sensor information captured at a particular altitude above the rooftop;

determining, by the one or more processors of the UAV executing a visual classifier trained to identify rooftop damage and using the obtained first sensor information, that the particular area of the rooftop is indicated as being damaged;

determining a corresponding location associated with the particular area of the rooftop indicated as being damaged, and navigating the UAV to a position above the corresponding location; and

descending, from the position, towards the particular area and obtaining second sensor information periodically while descending, wherein descending comprises:

performing visual odometry using the obtained first sensor information and second sensor information, and

correcting a flight path of the UAV, along a horizontal direction, while descending based on the performed visual odometry, such that the UAV maintains descent along a vertical direction from the position towards the particular area.

29. The method of claim 28 , wherein correcting a flight path of the UAV while descending comprises:

determining, from the first sensor information, a center of the damage included in the particular area; and

correcting, using the second sensor information, the flight path along the horizontal direction to maintain the descent towards the center of the damage.

30. The method of claim 28 , wherein the visual classifier determines rooftop damage based on one or more features included in sensor information that are indicative of rooftop damage, wherein a feature comprises: cracks in ceramic/clay tiles, broken corners of tiles, gaps between shingles, curled edges of a rooftop, dents in a wooden rooftop, dents or divots in tiles, moss or algae on shingles, shingle splitting, surface nail damage, spatter marks, or nails raised above a surface of the rooftop.

Assignments (11)