Vehicle-mounted control unit, and method and apparatus for FPGA based automatic driving of vehicle
Embodiments of the present disclosure provide a vehicle-mounted control unit, and a method and an apparatus for FPGA based automatic driving of a vehicle, which includes a MCU and a first SoC implemented by being integrated with an ARM through the FPGA, where the vehicle-mounted control unit is set on an automatic driving vehicle, the FPGA of the first SoC receives video data sent by a vehicle-mounted camera, performs visual perception on the video data by using a first neural network algorithm to obtain first perception information; and sends the first perception information to the ARM of the first SoC. The ARM of the first SoC processes the first perception information to obtain first decision information, and sends the first decision information to the MCU. Finally, the MCU generates a control command according to the first decision information and sends it to the corresponding execution mechanism.
1. A method for field programmable gate array (FPGA) based automatic driving of a vehicle, suitable for a vehicle-mounted control unit, wherein the vehicle-mounted control unit comprises a first system on chip (SoC), a second SoC and a micro control unit (MCU),
wherein the first SoC is integrated with a first advanced reduced instruction set computer machine (ARM) through a first FPGA, the first FPGA and the first ARM are encapsulated in the first SoC, and the vehicle-mounted control unit is set on an automatic driving vehicle, wherein the method comprises:
receiving, by the first FPGA of the first SoC, video data sent by a vehicle-mounted camera;
performing, by the first FPGA of the first SoC, visual perception on the video data by using a first neural network algorithm to obtain first perception information;
sending, by the first FPGA of the first SoC, the first perception information to the first ARM of the first SoC; and
processing, by the first ARM of the first SoC, the first perception information to obtain first decision information, and sending the first decision information to the MCU;
wherein the second SoC is integrated with a second ARM through a second FPGA, the second FPGA and the second ARM are encapsulated in the second SoC, and the method further comprises:
receiving, by the second FPGA of the second SoC, the video data sent by the vehicle-mounted camera;
performing, by the second FPGA of the second SoC, visual perception on the video data by using a second neural network algorithm to obtain second perception information;
sending, by the second FPGA of the second SoC, the second perception information to the second ARM of the second SoC;
processing, by the second ARM of the second SoC, the second perception information to obtain second decision information, and sending the second decision information to the MCU; and
generating, by the MCU, a control command according to the first decision information and the second decision information;
wherein generating, by the MCU, the control command according to the first decision information and the second decision information comprises:
comparing and analyzing, by the MCU, the first decision information and the second decision information to select an appropriate strategy information; and
generating, by the MCU, the control command according to the appropriate strategy information selected.
2. The method according to claim 1 , wherein the processing, by the ARM of the first SoC, the first perception information to obtain first decision information, and sending the first decision information to the MCU comprises:
receiving radar data by the ARM of the first SoC;
merging, by the ARM of the first SoC, the first perception information and the radar data; and
processing, by the ARM of the first SoC, the merged first perception information and the radar data to obtain the first decision information, and sending the first decision information to the MCU.
3. The method according to claim 2 , wherein the radar data comprises at least one of ultrasonic radar data, millimeter wave radar data, and lidar data.
4. A vehicle-mounted control unit, comprising: a first system on chip (SoC), a second SoC, and a micro control unit (MCU), wherein the first SoC and the second SoC are respectively integrated with an advanced reduced instruction set computer machinate array (ARM) through a field programmable gate array (FPGA), and the first SoC and the second SoC are connected to the MCU through an Ethernet switching chip;
wherein the first SoC comprises a first FPGA unit and a first ARM unit, and the first unit FPGA and the first ARM unit are encapsulated in the first SoC,
the first FPGA unit is configured to receive video data sent by a vehicle-mounted camera, perform visual perception on the video data by using a first neural network algorithm to obtain first perception information, and send the first perception information to the first ARM unit of the first SoC; and
the first ARM unit is configured to process the first perception information to obtain first decision information, and send the first decision information to the MCU;
wherein the second SoC comprises a second FPGA unit and a second ARM unit, and the second FPGA unit and the second ARM unit are encapsulated in the second SoC,
the second FPGA unit is configured to receive the video data sent by the vehicle-mounted camera, perform visual perception on the video data by using a second neural network algorithm to obtain second perception information, and send the second perception information to the second ARM unit of the second SoC; and
the second ARM unit is configured to process the second perception information to obtain second decision information, and send the second decision information to the MCU;
wherein the MCU is configured to compare and analyze the first decision information and the second decision information to select an appropriate strategy information;
the MCU is further configured to generate a control command according to the appropriate strategy information selected.
5. The vehicle-mounted control unit according to claim 4 , wherein the first ARM unit is configured to receive radar data, merge the first perception information and the radar data, process the merged first perception information and the radar data to obtain first decision information, and send the first decision information to the MCU.
6. The vehicle-mounted control unit according to claim 5 , wherein the radar data comprises at least one of ultrasonic radar data, millimeter wave radar data, and lidar data.
7. The vehicle-mounted control unit according to claim 4 , further comprising:
a first synchronous dynamic random access memory (SDRAM) and a first flash memory (Flash), wherein the first SDRAM is connected to the first SoC, and the first Flash is connected to the first SoC.
8. An automatic driving apparatus, wherein the apparatus is suitable for a vehicle-mounted control unit, and the automatic driving apparatus comprises: a first system on chip (SoC) module, a second SoC module, and a micro control unit (MCU) module, the first SoC module and the second SoC module are respectively integrated with an advanced reduced instruction set computer machine (ARM) through a field programmable gate array (FPGA), the vehicle-mounted control unit is set on an automatic driving vehicle,
wherein the first SoC module comprises a first FPGA unit and a first ARM unit, and the first FPGA unit and the first ARM unit are encapsulated in the first SoC module,
the first FPGA unit is configured to receive video data sent by a vehicle-mounted camera, perform visual perception on the video data by using a first neural network algorithm to obtain first perception information, and send the first perception information to the first ARM unit of the first SoC module; and
the first ARM unit is configured to process the first perception information to obtain first decision information, and send the first decision information to the MCU module;
wherein the second SoC module comprises a second FPGA unit and a second ARM unit, and the second FPGA unit and the second ARM unit are encapsulated in the second SoC module,
the second FPGA unit is configured to receive the video data sent by the vehicle-mounted camera, perform visual perception on the video data by using a second neural network algorithm to obtain second perception information, and send the second perception information to the second ARM unit of the second SoC module; and
the second ARM unit is configured to process the second perception information to obtain second decision information, and send the second decision information to the MCU module;
wherein the MCU module is configured to compare and analyze the first decision information and the second decision information to select an appropriate strategy information;
the MCU module is further configured to generate a control command according to the appropriate strategy information selected.
9. The apparatus according to claim 8 , wherein the first ARM unit is configured to receive radar data, merge the first perception information and the radar data, process the merged first perception information and the radar data to obtain first decision information, and send the first decision information to the MCU.
10. The apparatus according to claim 8 , wherein the radar data comprises at least one of ultrasonic radar data, millimeter wave radar data, and lidar data.