Granted Patent B2

US 12,266,169 · App. 17/810,948 · Granted Apr 1, 2025

Inventors: Yan Yang (Brisbane, AU); Paul Foster (Brisbane, AU)

Assignee: The Boeing Company

G06V20/17B64C39/024G06V10/22G06V10/757B64U2101/30G06V2201/07

Abstract

Claims (67)

1. A computer-implemented method for automated change assessment of target objects, the method comprising using a number of processors to perform operations of:

recording an aerial image of a target area during a first flyover;

detecting a target object in the aerial image with a corresponding confidence score above a detection threshold;

recording a sequence of aerial images of the target area during a subsequent second flyover;

calculating a transition probability of consecutive image frames in the sequence of aerial images based on a probability of change from one frame to another;

applying a penalty score to the transition probability in favor of object detection;

updating a percentage of change and the probability of change in real-time according to the transition probability and a status of change in the current and all previous frames in the sequence of aerial images;

determining in real-time a probability of detection of the target object according to respective confidence scores of the sequence of aerial images;

determining in real-time a status of change of the target object according to the confidence scores of the sequence of aerial images;

responsive to a determination that the target object has changed, determining by feature matching in real-time a percentage of change in a distance between an area of interest in the aerial image from the first flyover and each image of the sequence of aerial images from the second flyover;

combining the probability of detection of the target object and the percentage of change as statistically independent events to determine a probability of change; and

outputting in real-time the probability of change and percentage of change for each image in the sequence of images.

2. The method of claim 1 , further comprising recording a geographical position of the target object.

3. The method of claim 2 , further comprising outputting a final change assessment according to the percentage of change of the target object when a flight camera field of view exits a geographical boundary of the target area.

4. The method of claim 1 , wherein determining the percentage change of the target object further comprises hidden Markov chain normalization of the percentage of change of the target object.

5. The method of claim 1 , wherein the method is performed onboard an unmanned aerial vehicle.

6. The method of claim 1 , further comprising determining a geographical boundary of the target area based on a geographical position of the target object and a flight camera field of view, wherein the geographical boundary of the target area is used during the second flyover to ensure the geographical position of the target object stays within corner pixels of each of the sequence of aerial images.

7. The method of claim 1 , wherein the feature matching comprises scale and rotation invariant key point matching.

8. A computer-implemented method for real-time automated change assessment by a flight computer onboard an unmanned aerial vehicle, the method comprising performing, using a number of processors onboard the unmanned aerial vehicle, operations of:

recording, with an onboard flight camera, an aerial image of a target area during a first flyover;

detecting a target object in the aerial image with a corresponding confidence score above a detection threshold;

recording, with the onboard flight camera, a sequence of aerial images of the target area during a subsequent second flyover;

calculating a transition probability of consecutive image frames in the sequence of aerial images based on a probability of change from one frame to another;

applying a penalty score to the transition probability in favor of object detection;

updating a percentage of change and probability of change in real-time according to the transition probability and a status of change in the current and all previous frames in the sequence of aerial images;

determining in real-time a probability of detection of the target object according to respective confidence scores of the sequence of aerial images;

determining in real-time a status of change of the target object according to the confidence scores of the sequence of aerial images;

responsive to a determination that the target object has changed, determining by image feature matching in real-time a percentage of change in a distance between an area of interest in the aerial image from the first flyover and each image of the sequence of aerial images from the second flyover;

combining the probability of detection of the target object and the percentage of change as statistically independent events to determine a probability of change; and

outputting in real-time the probability of change and percentage of change for each image in the sequence of images.

9. The method of claim 8 , further comprising recording a geographical position of the target object.

10. The method of claim 9 , further comprising outputting a final change assessment according to the percentage of change of the target object when the unmanned aerial vehicle exits a geographical boundary of the target area.

11. The method of claim 8 , wherein determining the percentage change of the target object further comprises hidden Markov chain normalization of the percentage of change of the target object.

12. The method of claim 8 , further comprising determining a geographical boundary of the target area based on a geographical position of the target object and a field of view of the onboard flight camera, wherein the geographical boundary of the target area is used during the second flyover to ensure the geographical position of the target object stays within corner pixels of each of the sequence of aerial images.

13. A system configured to assess change of target objects, wherein the system comprises:

a storage device configured to store program instructions; and

one or more processors operably connected to the storage device and configured to execute the program instructions to cause the system to:

record an aerial image of a target area during a first flyover;

detect a target object in the aerial image with a corresponding confidence score above a detection threshold;

record a sequence of aerial images of the target area during a subsequent second flyover;

calculate a transition probability of consecutive image frames in the sequence of aerial images based on a probability of change from one frame to another;

apply a penalty score to the transition probability in favor of object detection;

update a percentage of change and probability of change in real-time according to the transition probability and a status of change in the current and all previous frames in the sequence of aerial images;

determine in real-time a probability of detection of the target object according to respective confidence scores of the sequence of aerial images;

determine in real-time a status of change of the target object according to the confidence scores of the sequence of aerial images;

responsive to a determination that the target object has changed, determine based upon image feature matching in real-time a percentage of change in a distance between an area of interest in the aerial image from the first flyover and each image of the sequence of aerial images from the second flyover;

combine the probability of detection of the target object and the percentage of change as statistically independent events to determine a probability of change; and

output in real-time the probability of change and percentage of change for each image in the sequence of images.

14. The system of claim 13 , wherein the processors further execute instructions to record a geographical position of the target object.

15. The system of claim 14 , wherein the processors further execute instructions to output a final change assessment according to the percentage of change of the target object when a flight camera field of view exits a geographical boundary of the target area.

16. The system of claim 13 , wherein determining the percentage change of the target object further comprises hidden Markov chain normalization of the percentage of change of the target object.

17. The system of claim 13 , wherein the system is on board an unmanned aerial vehicle.

18. The system of claim 13 , wherein the processors further execute instructions to determine a geographical boundary of the target area based on a geographical position of the target object and a flight camera field of view, wherein the geographical boundary of the target area is used during the second flyover to ensure the geographical position of the target object stays within corner pixels of each of the sequence of aerial images.

19. The system of claim 13 , wherein the image feature matching comprises scale and rotation invariant key point matching.

20. A computer program product configured to assess change target objects, wherein the computer program product comprises a non-transitory computer-readable storage medium comprises program instructions embodied thereon configured to perform the steps of:

recording an aerial image of a target area during a first flyover;

detecting a target object in the aerial image with a corresponding confidence score above a detection threshold;

recording a sequence of aerial images of the target area during a subsequent second flyover;

calculating a transition probability of consecutive image frames in the sequence of aerial images based on a probability of change from one frame to another;

applying a penalty score to the transition probability in favor of object detection;

updating a percentage of change and probability of change in real-time according to the transition probability and a status of change in the current and all previous frames in the sequence of aerial images;

determining in real-time a probability of detection of the target object according to respective confidence scores of the sequence of aerial images;

determining in real-time a status of change of the target object according to the confidence scores of the sequence of aerial images;

responsive to a determination that the target object has changed, determining by image feature matching in real-time a percentage of change in a distance between an area of interest in the aerial image from the first flyover and each image of the sequence of aerial images from the second flyover;

combining the probability of detection of the target object and the percentage of change as statistically independent events to determine a probability of change;

outputting in real-time the probability of change and percentage of change for each image in the sequence of images; and

outputting a final change assessment upon leaving geographical boundaries of the target area.

Assignments (1)

Continuity (1)

References Cited (39)