Bird detection and species determination
Methods of determining the species of birds in flight are provided along with corresponding systems. A method may include capturing a video stream of a bird in flight using at least one camera, generating a first species probability estimate by delivering images from the video stream to a neural network that has been trained to recognize species of birds from images, obtaining additional parameters from the video stream or from additional data, generating a second species probability estimate by delivering the additional parameters as input to a domain knowledge module with a domain knowledge statistical model, and generating a final species probability estimate by combining the first species probability estimate and the second species probability estimate. The additional parameters may include geometry features related to movement of the bird in flight, or parameters relating to the environment.
1 . A method of determining the species of birds in flight, comprising:
capturing at least one video stream of a bird in flight using at least one camera;
generating a first species probability estimate by delivering images from the at least one video stream as input to an artificial neural network that has been trained to recognize species of birds from images;
delivering images from the at least one video stream as input to a computer-executed geometry feature extraction software module that outputs extracted geometric features related to the bird in flight in the at least one video stream;
delivering the extracted geometric features outputted from the computer-executed geometry extraction software module as input to a computer-executed domain knowledge software module utilizing a domain knowledge statistical model to output a second species probability estimate based on the extracted geometric features; and
generating a final species probability estimate by combining the first species probability estimate and the second species probability estimate,
wherein the extracted geometric features outputted from the computer-executed geometry extraction software module are selected from the group consisting of: size, positions, acceleration, vertical motion, flight trajectory, and wingbeat frequency.
2 . The method according to claim 1 , further comprising:
generating the first species probability estimate by delivering extracted features from the artificial neural network and the extracted geometric features outputted from the computer-executed geometry feature extraction software module as input to a shallow neural network that has been trained to generate bird species probabilities based on features extracted by an artificial neural network combined with observed geometric features.
3 . The method according to claim 2 , wherein the extracted geometric features are obtained based on identification of the same bird in a sequence of images from the at least one video stream, and estimating motion based on the change of the identified bird's position between images in the sequence of images.
4 . The method according to claim 2 , wherein the at least one camera is two or more cameras and the at least one video stream is two or more video streams;
wherein the extracted geometric features are obtained based on a known position of each camera, identification of the same bird in two or more sequences of images from two or more concurrent video streams, determination of the position of the identified bird in the respective images of the respective video streams, and using multi-view geometry analysis to determine 3D coordinates representative of positions of the identified bird relative to the positions of the cameras from the determined positions in the respective images of the respective video streams.
5 . The method according to claim 2 , wherein one extracted geometric feature is a wingbeat frequency determined by performing Fourier analysis on a sequence of images from the at least one video stream, and identifying a dominant frequency component that is inside a frequency interval consistent with wingbeat frequencies for birds.
6 . The method according to claim 1 , further comprising:
training the artificial neural network by delivering a dataset including labeled images of relevant bird species as input to the artificial neural network.
7 . The method according to claim 1 , further comprising:
performing object detection on images from the at least one video stream and annotating the images with bounding boxes drawn around each object that is identified as a bird.
8 . The method according to claim 7 , wherein object detection is performed using a second artificial neural network.
9 . The method according to claim 1 , further comprising:
providing the species with the highest determined final species probability as output.
10 . The method according to claim 9 , further comprising using the output to control a means of deterrent or curtailment in order to reduce a risk that the bird of the determined species is injured by a wind farm installation.
11 . The method according to claim 1 , wherein the domain knowledge statistical model is a Bayesian belief network and/or one or more artificial neural networks are convolutional neural networks.
12 . A system for determining the species of birds in flight, comprising:
at least one video camera;
one or more computer processors programmed to execute
an artificial neural network configured to receive video images from the at least one video camera and trained to recognize species of birds from images;
a geometry feature extraction software module configured to receive at least one video stream from the at least one video camera and to output extracted geometric features related to birds captured in flight in the at least one video stream;
a domain knowledge software module with a domain knowledge statistical model, configured to receive the extracted geometric features outputted by the geometry feature extraction software module and to generate a probability of observing respective species of birds given the extracted geometric features; and
a species determination software module configured to receive a first species probability estimate based on output from the artificial neural network and a second species probability estimate based on output from the domain knowledge software module and to generate a final species probability estimate,
wherein the extracted geometric features outputted from the computer-executed geometry extraction software module are selected from the group consisting of: size, positions, acceleration, vertical motion, flight trajectory, and wingbeat frequency.
13 . The system according to claim 12 , further comprising:
a shallow neural network configured to receive extracted features from the artificial neural network and extracted geometry features outputted from the geometry feature extraction software module, and to generate the first species probability estimate.
14 . The system according to claim 13 , wherein the geometry feature extraction software module is configured to received data related to at least one video stream, extract geometric features based on identification of the same bird in a sequence of images from the at least one video stream, and to estimate motion based on the change of the identified bird's position between images in the sequence of images.
15 . The system according to claim 13 , wherein the at least one camera is two or more cameras and the at least one video stream is two or more video streams;
the one or more computer programs being programmed to further execute a multi-view geometry analysis software module configured to receive a known position of each camera, receive data related to at least two concurrent video streams, determine a position of a bird identified in the respective images of the respective video streams, determine the position of the identified bird in the respective images of the respective video streams, and use multi-view geometry analysis to determine 3D coordinates representative of positions of the identified bird relative to the positions of the cameras from the determined positions in the respective images of the respective video streams.
16 . The system according to claim 13 , wherein the geometry features extraction software module is further configured to determine a wingbeat frequency by performing Fourier analysis on a sequence of images from the at least one video stream and identifying a dominant frequency component that is inside a frequency interval consistent with wingbeat frequencies for birds.
17 . The system according to claim 12 , wherein the one or more computer processors is programmed to further execute a bird detection and tracking software module configured to receive input from at least one video camera and perform object detection and to annotate images by drawing bounding boxes around each object that is identified as a bird.
18 . The system according to claim 17 , wherein the bird detection and tracking software module includes a second artificial neural network.
19 . The system according to claim 12 , wherein the species determination software module is further configured to deliver the final species probability estimate as output to be stored, displayed, or used to control a process of deterrence or curtailment in order to reduce a risk that the bird of the determined species is injured by a wind farm installation.
20 . The system according to claim 12 , wherein the domain knowledge statistical model is a Bayesian belief network and/or one or more artificial neural networks are convolutional neural networks.