Object detection unit

1 . An object detection unit for detecting thea presence of foreign object debris (FOD) on a surface of a runway, taxiway or apron, the object detection unit comprising a processor, the object detection unit configured to:

receive an image taken by a camera, the image comprising at least part of the runway surface, taxiway surface or apron surface,

input the image to a zero-shot object detection model, wherein the zero-shot object detection model is configured to output co-ordinates of a bounding box around a candidate FOD in the image and determine a likelihood that FOD is present in the bounding box;

obtain, from the zero-shot object detection model, co-ordinates of a bounding box around a candidate FOD in the image and a likelihood that the candidate FOD is foreign object debris; and

determine if the likelihood exceeds a threshold value,

wherein when the likelihood exceeds the threshold value, the object detection unit is further configured to:

locate a center of the bounding box based on the co-ordinates of the bounding box;

trace a ray from a focal point of the camera to the center of the bounding box in an imaginary image plane of a sensor of the camera; and

determine a position of the candidate FOD on the surface based on an intersection of the ray with the surface.

2 . The object detection unit of claim 1 , wherein when the likelihood exceeds the threshold value, the object detection unit is configured to:

compare a region of the image inside the bounding box to a set of known objects; and either:

determine that the candidate FOD matches a known object and discard the bounding box; or

determine that the candidate FOD does not match a known object.

3 . The object detection unit of claim 1 , wherein the object detection unit is further configured to:

divide the image into a first set of pixels and a second set of pixels,

the first set of pixels comprising pixels that represent the runway, taxiway or apron,

the second set of pixels comprising pixels that represent regions of the image other than the runway, taxiway and apron; and

discard the bounding box when pixels in the bounding box consist of pixels in the second set of pixels.

4 . The object detection unit of claim 1 , wherein the foreign object debris comprises a plurality of objects, and the object detection unit is configured to:

output co-ordinates of a respective bounding box around each candidate FOD in the image,

wherein at least two bounding boxes around two separate candidate FODs intersect, and the object detection unit is further configured to:

replace the intersecting bounding boxes with one bounding box around the candidate FODs.

5 . The object detection unit of claim 2 , wherein when the object detection unit determines that the candidate FOD does not match a known object, the object detection unit is configured to:

determine that the candidate FOD is foreign object debris;

send an alert comprising metadata associated with the image to a control station of a vehicle for monitoring and reporting runway, taxiway and apron surface conditions;

draw, based on the co-ordinates of the bounding box, a bounding box around the foreign object debris in the image;

downscale regions of the image outside the bounding box to create a partially downscaled version of the image; and

send the partially downscaled version of the image to the control station of the vehicle.

6 . The object detection unit of claim 1 , wherein the object detection unit is further configured to:

split the image into a plurality of sub-images;

wherein inputting the image to the zero-shot object detection model comprises inputting each of the plurality of sub-images to the zero-shot object detection model.

7 . The object detection unit of claim 1 , wherein the object detection unit is further configured to:

scan the image for human faces;

locate a human face in the image; and

set values of pixels corresponding to the human face to a preset value.

8 . The object detection unit according to claim 1 , wherein the zero-shot object detection model is trained on a set of training images,

wherein after training the zero-shot object detection model the zero-shot object detection model is configured to detect unseen instances of FOD,

wherein unseen instances of FOD comprise instances of FOD absent in the set of training images.

9 . The object detection unit of claim 1 , wherein the zero-shot object detection model comprises an open vocabulary detection model, wherein a text prompt is provided to the open vocabulary detection model, the text prompt comprising a description of an object, and the open vocabulary detection model is configured to detect an object in the image that matches the description of the object.

10 . A vehicle for monitoring runway, taxiway and apron surface conditions and remotely reporting a status of the runway, taxiway or apron, the vehicle comprising a detection unit, and a communications unit,

the detection unit comprising:

an object detection unit;

a processor; and

a camera configured to obtain an image of at least part of the surface, the object detection unit comprising a processor, the object detection unit configured to:

receive an image taken by the camera, the image comprising at least part of the runway surface, taxiway surface or apron surface,

input the image to a zero-shot object detection model, wherein the zero-shot object detection model is configured to output co-ordinates of a bounding box around a candidate FOD in the image and determine a likelihood that FOD is present in the bounding box;

obtain, from the zero-shot object detection model, co-ordinates of a bounding box around a candidate FOD in the image and a likelihood that the candidate FOD is foreign object debris; and

determine if the likelihood exceeds a threshold value,

wherein when the likelihood exceeds the threshold value, the object detection unit is configured to:

compare a region of the image inside the bounding box to a set of known objects; and either:

determine that the candidate FOD matches a known object and discard the bounding box; or

determine that the candidate FOD does not match a known object, and

wherein when the object detection unit determines that the candidate FOD does not match a known object, the object detection unit is configured to:

determine that the candidate FOD is foreign object debris;

send an alert comprising metadata associated with the image to a control station of a vehicle for monitoring and reporting runway, taxiway and apron surface conditions;

draw, based on the co-ordinates of the bounding box, a bounding box around the foreign object debris in the image;

downscale regions of the image outside the bounding box to create a partially downscaled version of the image; and

send the partially downscaled version of the image to the control station of the vehicle.

11 . A method of monitoring aerodrome runway, taxiway and apron conditions, the method performed by an object detection unit, the method comprising the steps of:

receiving an image taken by a camera, the image comprising at least part of a surface of the runway, taxiway or apron;

inputting the image to a zero-shot object detection model, wherein the zero-shot object detection model is configured to output co-ordinates of a bounding box around a candidate FOD in the image and determine a likelihood that FOD is present in the bounding box;

obtaining, from the zero-shot object detection model, co-ordinates of a bounding box around a candidate FOD in the image and a likelihood that the candidate FOD is foreign object debris; and

determining if the likelihood exceeds a threshold value,

wherein when the likelihood exceeds the threshold value, the method further comprises:

locating a center of the bounding box based on the co-ordinates of the bounding box;

tracing a ray from a focal point of the camera to the center of the bounding box in an imaginary image plane of a sensor of the camera; and

determining a position of the candidate FOD on the surface based on an intersection of the ray with the surface.

12 . The method of claim 11 , wherein when the likelihood exceeds the threshold value, the method further comprises:

comparing a region of the image inside the bounding box to a set of known objects; and either:

determining that the candidate FOD matches a known object and discard the bounding box; or

determining that the candidate FOD does not match a known object.

13 . The method according to claim 11 , further comprising:

dividing the image into a first set of pixels and a second set of pixels,

the first set of pixels comprising pixels that represent the runway, taxiway or apron,

the second set of pixels comprising pixels that represent regions of the image other than the runway, taxiway and apron; and

discarding the bounding box when pixels in the bounding box consist of pixels in the second set of pixels.

14 . The method of claim 11 , wherein the foreign object debris comprises a plurality of objects, and the method further comprises:

outputting co-ordinates of a respective bounding box around each candidate FOD in the image,

wherein at least two bounding boxes around two separate candidate FODs intersect; and

replacing the intersecting bounding boxes with one bounding box around the candidate FODs.

15 . The method of claim 12 , wherein when the candidate FOD does not match a known object, the method further comprises:

determining that the candidate FOD is foreign object debris;

sending an alert comprising metadata associated with the image to a control station of a vehicle for monitoring and reporting runway, taxiway and apron surface conditions;

drawing, based on the co-ordinates of the bounding box, a bounding box around the foreign object debris in the image;

downscaling regions of the image outside the bounding box to create a partially downscaled version of the image; and

sending the partially downscaled version of the image to the control station of the vehicle.

16 . The method of claim 11 , further comprising:

splitting the image into a plurality of sub-images;

wherein inputting the image to the zero-shot object detection model comprises inputting each of the plurality of sub-images to the zero-shot object detection model.

17 . The method of claim 11 , further comprising:

scanning the image for human faces;

locating a human face in the image; and

setting values of pixels corresponding to the human face to a preset value.

18 . The method of claim 11 , wherein the zero-shot object detection model comprises an open vocabulary detection model, wherein a text prompt is provided to the open vocabulary detection model, the text prompt comprising a description of an object, and the open vocabulary detection model is configured to detect an object in the image that matches the description of the object.

19 . A method of monitoring aerodrome runway, taxiway and apron conditions, the method performed by an object detection unit, the method comprising the steps of:

receiving an image taken by a camera, the image comprising at least part of a surface of the runway, taxiway or apron;

inputting the image to a zero-shot object detection model, wherein the zero-shot object detection model is configured to output co-ordinates of a bounding box around a candidate FOD in the image and determine a likelihood that FOD is present in the bounding box;

obtaining, from the zero-shot object detection model, co-ordinates of a bounding box around a candidate FOD in the image and a likelihood that the candidate FOD is foreign object debris; and

determining if the likelihood exceeds a threshold value,

wherein when the likelihood exceeds the threshold value, the method further comprises:

comparing a region of the image inside the bounding box to a set of known objects; and either:

determining that the candidate FOD matches a known object and discard the bounding box; or

determining that the candidate FOD does not match a known object, and

wherein when the candidate FOD does not match a known object, the method further comprises:

determining that the candidate FOD is foreign object debris;

sending an alert comprising metadata associated with the image to a control station of a vehicle for monitoring and reporting runway, taxiway and apron surface conditions;

drawing, based on the co-ordinates of the bounding box, a bounding box around the foreign object debris in the image;

downscaling regions of the image outside the bounding box to create a partially downscaled version of the image; and

sending the partially downscaled version of the image to the control station of the vehicle.