Method and apparatus with complex-valued attention network

1. A processor-implemented method of performing an operation using a complex-valued attention network, the method comprising:

extracting a complex-valued attention weight from complex-valued input data using one or more convolution operations;

normalizing a magnitude of the complex-valued attention weight while preserving a phase of the complex-valued attention weight;

determining complex-valued attention data by applying the normalized complex- valued attention weight to the complex-valued input data using an element-wise multiplication such that dimensions of the complex-valued attention data correspond to dimensions of the complex-valued input data;

determining ego-motion information of a vehicle from the complex-valued attention data based on an ego-motion estimation model; and

controlling any one or any combination of a velocity, an acceleration, and steering of the vehicle based on the ego-motion information.

2. The method of claim 1 , wherein the determining of the complex-valued attention data comprises individually applying a real value and an imaginary value of the extracted complex-valued attention weight to real data and imaginary data of the complex-valued input data.

3. The method of claim 2 , wherein

the applying of the normalized complex-valued attention weight to the complex-valued input data comprises applying the normalized weight of the complex-valued attention weight to the real data and the imaginary data for each channel.

4. The method of claim 3 , wherein the applying of the normalized weight comprises:

applying a real value of the normalized weight of the complex-valued attention weight to a real input map of the complex-valued input data for each channel; and

applying an imaginary value of the normalized weight of the complex-valued attention weight to an imaginary input map of the complex-valued input data for each channel.

5. The method of claim 3 , wherein the normalizing of the magnitude of the complex-valued attention weight comprises:

determining phase information of the complex-valued attention weight for each channel; and

determining a bounded magnitude of the complex-valued attention weight within a threshold range for each channel.

6. The method of claim 5 , further comprising:

applying the determined phase information and the determined bounded magnitude to the complex-valued input data through the element-wise multiplication.

7. The method of claim 5 , wherein values of the normalized weight correspond to an inner region of a circle having a radius that is a threshold corresponding to the threshold range in a complex plane.

8. The method of claim 5 , wherein

the determining of the phase information comprises dividing the complex-valued attention weight by an absolute value of the complex-valued attention weight for each channel, and

the determining of the bounded magnitude comprises applying an activation function to the absolute value of the complex-valued attention weight for each channel.

9. The method of claim 1 , wherein the extracting of the complex-valued attention weight comprises:

determining the complex-valued attention weight by extracting a value indicating a real component and a value indicating an imaginary component from the complex-valued input data for each channel.

10. The method of claim 9 , wherein the determining of the complex-valued attention weight comprises:

performing pooling on a real representative value representing a real component and an imaginary representative value representing an imaginary component from the complex-valued input data for each channel;

generating downscaled data by applying a convolution operation, of the one or more convolution operations, of reducing a number of channels to a result of the pooling including the real component and the imaginary component; and

determining the complex-valued attention weight by applying a convolution operation, of the one or more convolution operations, of increasing a number of channels to the downscaled data.

11. The method of claim 1 , further comprising:

obtaining raw radar data based on a radar signal sensed by a radar sensor; and generating the

complex-valued input data by transforming the raw radar data.

12. The method of claim 11 , wherein the obtaining of the raw radar data comprises obtaining an angle-velocity map for each range channel as the complex-valued input data.

13. The method of claim 11 , further comprising:

for the determining of the ego-motion information, determining the ego-motion information of the radar sensor from the complex-valued attention data based on the ego-motion estimation model.

14. The method of claim 13 , wherein the determining of the ego-motion information comprises determining an acceleration with respect to at least one axis together with a velocity and an angular velocity of the radar sensor as the ego-motion information.

15. The method of claim 13 , wherein the determining of the ego-motion information comprises determining the ego-motion information from residual data between the complex-valued input data and the complex-valued attention data based on the ego-motion estimation model.

16. The method of claim 13 , wherein the radar sensor is mounte in the vehicle.

17. The method of claim 13 , further comprising:

determining at least one of a position and a heading direction of the vehicle, in which the radar sensor is mounted, based on the ego-motion information; and

outputting an estimation result for at least one of the position and the heading direction.

18. The method of claim 1 , further comprising:

determining residual data by summing the complex-valued input data and the complex- valued attention data, and applying a complex-valued attention network-based operation to the residual data.

19. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, configure the processor to perform the method of claim 1 .

20. A computing apparatus comprising:

a memory configured to store a complex-valued attention network model; and a processor configured to

extract a complex-valued attention weight from complex-valued input data using one or more convolution operations of a first layer of the complex-valued attention network model,

normalize a magnitude of the complex-valued attention weight while preserving a phase of the complex-valued attention weight,

determine complex-valued attention data by applying the normalized complex-valued attention weight to real data and imaginary data of the complex-valued input data using a second layer of the complex-valued attention network model and an element-wise multiplication such that dimensions of the complex-valued attention data correspond to dimensions of the complex-valued input data,

determine ego-motion information of a vehicle from the complex-valued attention data based on an ego-motion estimation model, and

control any one or any combination of a velocity, an acceleration, and steering of the vehicle based on the ego-motion information.

21. A radar data processing apparatus comprising:

a radar sensor configured to obtain raw radar data by radiating and receiving a radar signal; and

a processor configured to

generate complex-valued input data from the raw radar data,

extract a complex-valued attention weight from the complex-valued input data using one or more convolution operations,

normalize a magnitude of the complex-valued attention weight while preserving a phase of the complex-valued attention weight,

determine complex-valued attention data by applying the normalized complex-valued attention weight to real data and imaginary data of the complex-valued input data using an element-wise multiplication such that dimensions of the complex-valued attention data correspond to dimensions of the complex-valued input data,

determine ego-motion information of a vehicle from the complex-valued attention data based on an ego-motion estimation model, and

control any one or any combination of a velocity, an acceleration, and steering of the vehicle based on the ego-motion information.

22. A vehicle comprising:

a radar sensor configured to obtain raw radar data by radiating and receiving a radar signal; and

a processor configured to

generate complex-valued input data from the raw radar data,

extract a complex-valued attention weight from the complex-valued input data using one or more convolution operations,

normalize a magnitude of the complex-valued attention weight while preserving a phase of the complex-valued attention weight,

determine complex-valued attention data by applying the normalized complex-valued attention weight to real data and imaginary data of the complex-valued input data using an element-wise multiplication such that dimensions of the complex-valued attention data correspond to dimensions of the complex-valued input data,

determine ego-motion information of the vehicle from the complex-valued attention data based on an ego-motion estimation model, and

control any one or any combination of a velocity, an acceleration, and steering of the vehicle based on the ego-motion information.

23. A processor-implemented method of performing an operation using a complex-valued attention network, the method comprising:

generating complex-valued input data from raw radar data;

normalizing a magnitude of a complex-valued attention weight extracted using one or more convolution operations while preserving a phase of the complex-valued attention weight;

determining complex-valued attention data by applying, using an element-wise multiplication, the complex-valued input data to a complex-valued attention network model using the normalized complex-valued attention weight such that dimensions of the complex-valued attention data correspond to dimensions of the complex-valued input data;

determining ego-motion information based on the complex-valued attention data; and

controlling any one or any combination of a velocity, an acceleration, and steering of a vehicle based on the ego-motion information.

24. The method of claim 23 , wherein the determining of the complex-valued attention data comprises:

extracting the complex-valued attention weight from the complex-valued input data; and

determining the complex-valued attention data by applying the normalized complex-valued attention weight to the complex-valued input data.