Method of generating a peripheral image of an aircraft and associated electronic generation device and computer program product
A method of generating a peripheral image of an aircraft having pixels obtained from a plurality of cameras equipping the aircraft. The method is implemented by an electronic generation device. The method comprises a step of receiving a position of a peripheral point of view from which the peripheral image is to be generated. The method comprises steps of obtaining an orientation of the aircraft and of acquisition of a respective image from each camera. The method comprises a step of calculating a respective transfer function for each camera, each transfer function being intended for being applied to a pixel of the peripheral image so as to identify a possible corresponding pixel of the acquired image associated with the transfer function. The method comprises a step of generating the peripheral image from the images acquired from each camera and from each calculated transfer function.
1 . A method for generating a peripheral image of an aircraft, the peripheral image comprised of a plurality of peripheral pixels obtained from a plurality of cameras equipping the aircraft, the method:
receiving a position of a point of view from which the peripheral image is to be generated;
obtaining an orientation of the aircraft;
acquiring an image from each camera of the plurality of cameras equipping the aircraft, each acquired image comprising a plurality of acquired pixels;
calculating, for each respective camera of the plurality of cameras, a transfer function that maps peripheral pixels of the plurality of peripheral pixels to acquired pixels of the plurality of acquired pixels, each transfer function for a camera of the plurality of cameras thus being associated with the image acquired by that same camera, each transfer function mapping each peripheral pixel of the plurality of peripheral pixels of the peripheral image to a possible corresponding acquired pixel of the plurality of acquired pixels of the acquired image; and
generating the peripheral image from each of the acquired images and from each of the respective transfer functions,
wherein each transfer function of each camera of the plurality of cameras is of the form:
u
c
=
a
u
+
b
u
u
v
+
c
u
v
v
a
′
+
b
′
u
v
+
c
′
v
v
v
c
=
a
v
+
b
v
u
v
+
c
v
v
v
a
′
+
b
′
u
v
+
c
′
v
v
for each peripheral pixel of the plurality of peripheral pixels of the peripheral image,
where u c is an abscissa position of an acquired pixel of the plurality of acquired pixels of the image acquired by the camera, relative to a coordinate frame associated with the camera,
v c is an ordinate position of the acquired pixel of the plurality of acquired pixels of the image acquired by the camera, relative to the coordinate frame associated with the camera,
u v is an abscissa position of the peripheral pixel of the plurality of peripheral pixels of the peripheral image, relative to a reference coordinate frame,
v v is an ordinate position of the peripheral pixel of the plurality of peripheral pixels of the peripheral image, relative to the reference coordinate frame, and
a′, b′, c′, a u , b u , c u , a v , b v , c v are first to ninth coefficients, the first to ninth coefficients being specific to each camera of the plurality of cameras.
2 . The method according to claim 1 , wherein said obtaining comprises obtaining an orientation angle comprising at least one of a roll angle of the aircraft, a bearing angle of the aircraft, and a pitch angle of the aircraft.
3 . The method according to claim 1 , wherein each transfer function, when applied to a peripheral pixel of the plurality of peripheral pixels of the peripheral image, determines:
a position of an acquired pixel of the plurality of acquired pixels in the acquired image associated with the transfer function, if the peripheral pixel of the plurality of peripheral pixels of the peripheral image corresponds to an acquired pixel of the plurality of acquired pixels of the acquired image; or
an incorrect result, otherwise.
4 . The method according to claim 1 , wherein said generating comprises determining a color of each peripheral pixel of the plurality of peripheral pixels of the peripheral image by applying, for each peripheral pixel, each transfer function to that peripheral pixel and a rule for mixing the acquired images for that peripheral pixel.
5 . The method according to claim 4 , wherein the rule for mixing the images for a given peripheral pixel of the plurality of peripheral pixels of the peripheral image is:
if applying each transfer function to the given peripheral pixel of the plurality of peripheral pixels of the peripheral image provides the position of an acquired pixel of the plurality of acquired pixels in a single acquired image, then the color of that peripheral pixel of the peripheral image is equal to the color of that acquired pixel of the plurality of acquired pixels of the single acquired image;
if applying each transfer function to the given peripheral pixel of the plurality of peripheral pixels of the peripheral image provides acquired pixels of the plurality of acquired pixels for a plurality of the acquired images, then the color of that peripheral pixel of the peripheral image is equal to the color of the acquired pixel of the plurality of acquired pixels of that acquired image for which the acquired pixel of the plurality of acquired pixels is most central therein; and
otherwise, the color of the given peripheral pixel of the peripheral image is equal to a predefined color.
6 . The method according to claim 1 , further comprising sending the peripheral image to a display device, for being displayed to a pilot of the aircraft.
7 . The method according to claim 1 , further comprising adding a symbol representing the aircraft to the generated peripheral image, comprising:
determining a size and of a position of the symbol representing the aircraft, depending on the position of the point of view; and
including, in the peripheral image, the symbol representing the aircraft.
8 . The method according to claim 1 , wherein said obtaining, said acquiring, said calculating and said generating are reiterated a plurality of times, the plurality of thus-generated peripheral images forming a video stream.
9 . The method according to claim 1 , wherein said calculating comprises determining the first to ninth coefficients for each transfer function using that, for any acquired pixel of the plurality of acquired pixels appearing in the respective acquired image mapped by the transfer function from a peripheral pixel of the plurality of peripheral pixels in the peripheral image, the vector linking an origin of the coordinate frame associated with the camera and the acquired pixel of the plurality of acquired pixels passes through the point with the following
(
f
c
u
c
v
c
)
in the coordinate frame associated with the camera,
where:
f c is a focal length of the camera, and
u c and v c are abscissa and ordinate positions of the acquired pixel of the plurality of acquired pixels.
10 . The method according to claim 9 , wherein the coordinate frame associated with each camera comprises a first camera axis, a second camera axis and a third camera axis, perpendicular to each other, and wherein said calculating a transfer function comprises, for each respective camera:
calculating a rotation matrix between the coordinate frame associated with the camera and the reference coordinate frame, from the orientation of the aircraft;
calculating, for any peripheral pixel of the plurality of peripheral pixels in the peripheral image:
a first quantity representative of a contribution, along the first camera axis, of the vector linking the origin of the coordinate frame associated with the camera and the corresponding acquired pixel of the plurality of acquired pixels,
a second quantity representative of a contribution, along the second camera axis, of the vector linking the origin of the coordinate frame associated with the camera and the corresponding acquired pixel of the plurality of acquired pixels, and
a third quantity representative of a contribution, along the third camera axis, of the vector linking the origin of the coordinate frame associated with the camera and the corresponding acquired pixel of the plurality of acquired pixels,
each of the first, second, and third quantities depending on the calculated rotation matrix; and
determining the first, second and third coefficients from the first quantity, the fourth, fifth and sixth coefficients from the second quantity, and the seventh, eighth and ninth coefficients from third quantity, respectively.
11 . A non-transitory computer program product including software instructions which, when executed by a computer, cause the computer to implement a method according to claim 1 .
12 . An electronic device for generating a peripheral image of an aircraft, the peripheral image comprised of a plurality of peripheral pixels obtained from a plurality of cameras equipping the aircraft, the electronic device comprising:
a receiver receiving a position of a point of view from which the peripheral image is to be generated;
an obtainer obtaining an orientation of the aircraft;
an acquirer acquiring an image from each respective camera of the plurality of cameras equipping the aircraft, each acquired image comprising a plurality of acquired pixels;
a calculator calculating, for each camera of the plurality of cameras, a respective transfer function that maps peripheral pixels of the plurality of peripheral pixels to acquired pixels of the plurality of acquired pixels, each transfer function for a camera of the plurality of cameras thus being associated with the image acquired by that same camera, each transfer function mapping each peripheral pixel of the plurality of peripheral pixels of the peripheral image to a possible corresponding acquired pixel of the plurality of acquired pixels in the respective acquired image, wherein each transfer function of each camera of the plurality of cameras is of the form:
u
c
=
a
u
+
b
u
u
v
+
c
u
v
v
a
′
+
b
′
u
v
+
c
′
v
v
v
c
=
a
v
+
b
v
u
v
+
c
v
v
v
a
′
+
b
′
u
v
+
c
′
v
v
for each peripheral pixel of the plurality of peripheral pixels of the peripheral image,
where u c is an abscissa position of an acquired pixel of the plurality of acquired pixels of the image acquired by the camera, relative to a coordinate frame associated with the camera,
v c is an ordinate position of the acquired pixel of the plurality of acquired pixels of the image acquired by the camera, relative to the coordinate frame associated with the camera,
u v is an abscissa position of the peripheral pixel of the plurality of peripheral pixels of the peripheral image, relative to a reference coordinate frame,
V v is an ordinate position of the peripheral pixel of the plurality of peripheral pixels of the peripheral image, relative to the reference coordinate frame, and
a′, b′, c′, a u , b u , c u , a v , b v , c v are first to ninth coefficients, the first to ninth coefficients being specific to each camera of the plurality of cameras; and
a generator generating the peripheral image from each acquired image and from each respective transfer function.