Method and system for in-line data conversion outside of a machine learning hardware
A system includes a component configured to send data in a first data format. The system includes a direct memory access (DMA) engine configured to receive the data in the first data format and convert the first data format to a second data format, wherein the second data format is associated with a data format of a machine learning (ML) hardware, wherein the second data format is different from the first data format. The ML hardware is configured to receive the data in the second format and perform at least one ML operation on the received data in the second format. The received data in the second data format is stored on an on-chip memory (OCM) of the ML hardware.
1 . A system, comprising:
a memory component configured to store data in a first data format, wherein the memory component is external to a machine learning (ML) hardware;
a data streaming engine configured to stream data from the memory component external to the ML hardware to a memory component within the ML hardware, wherein the data streaming engine is configured to convert the data from the first data format to a second data format without storing the converted data in the second data format prior to transmitting the data in the second format to the ML hardware, wherein the second data format is associated with a data format of the ML hardware, and wherein the second data format is different from the first data format; and
said ML hardware configured to receive the data in the second format and wherein the ML hardware is configured to perform at least one ML operation on the received data in the second format.
2 . The system of claim 1 , wherein the data streaming engine comprises a data format conversion block configured to convert the data from the first data format to the second data format.
3 . The system of claim 1 , wherein the memory component external to the ML hardware is configured to store the data in the first data format as being received by a network interface card (NIC).
4 . The system of claim 3 , wherein the data is stored in the memory component external to the ML hardware via a direct memory access (DMA) engine.
5 . The system of claim 1 , wherein the memory component external to the ML hardware is a double data rate (DDR) memory.
6 . The system of claim 1 , wherein the first data format and the second data format is one of floating point (FP) 32, FP16, integer (INT) 8, unsigned int (UINT) 8, FP8, Brain FP (BF) 16, Fixed Point (FXP), In-phase Quadrature FP (IQFP), Quadrature (Q) format, quantization/dequantization, and scaling.
7 . The system of claim 1 , wherein said ML hardware is configured to transmit processed data in a third data format to the data streaming engine, and wherein the data streaming engine is configured to convert the processed data from the third data format to a fourth data format.
8 . The system of claim 7 , wherein the third data format is a same as the second data format, and wherein the first data format is a same as the fourth data format.
9 . A method comprising:
receiving data in a first data format;
converting the received data from the first data format to a second data format, wherein a machine learning (ML) hardware is configured to perform one or more operations based on the second data format, wherein the second data format is different from the first data format, and wherein the converting is performed by a hardware component;
transmitting the converted data in the second format to the ML hardware;
performing at least one ML operation associated with the data in the second format using the ML hardware; and
transmitting processed data from the ML hardware in a third data format to the hardware component, and wherein the hardware component is configured to convert the processed data from the third data format to a fourth data format, wherein the third data format is a same as the second data format, and wherein the first data format is a same as the fourth data format.
10 . The method of claim 9 , wherein the data is converted from the first data format to the second data format using a direct memory access (DMA) engine.
11 . The method of claim 10 , wherein the data is converted from the first data format to the second data format using the DMA engine as data is being received, and wherein the data in the second format is stored by the DMA engine in a memory component external to the ML hardware.
12 . The method of claim 9 , wherein the data is converted from the first data format to the second data format using the DMA engine as data is being fetched from a memory component that stored the received data in the first format for transmission to the ML hardware.
13 . The method of claim 12 , wherein the memory component is a double data rate (DDR) memory.
14 . The method of claim 9 , wherein the data is converted from the first data format to the second data format using a data streaming engine, wherein the data streaming engine is configured to stream data from a memory component external to the ML hardware that stores the received data in the first data format and converts the data from the first data format to the second data format without storing the converted data in the second data format prior to transmitting the data in the second format to the ML hardware.
15 . The method of claim 9 , wherein the first data format and the second data format is one of floating point (FP) 32, FP16, integer (INT) 8, unsigned int (UINT) 8, FP8, Brain FP (BF) 16, Fixed Point (FXP), In-phase Quadrature FP (IQFP), Quadrature (Q) format, quantization/dequantization, and scaling.
16 . The method of claim 9 further comprising storing the converted data in the second format within a memory component of the ML hardware.
17 . A system comprising:
a means for receiving data in a first data format;
a means for converting the received data from the first data format to a second data format, wherein a machine learning (ML) hardware is configured to perform one or more operations based on the second data format, wherein the second data format is different from the first data format, and wherein the converting is performed by a hardware component;
a means for transmitting the converted data in the second format to the ML hardware;
a means for performing at least one ML operation associated with the data in the second format using the ML hardware;
a means for transmitting processed data from the ML hardware in a third data format to the hardware component, and wherein the hardware component is configured to convert the processed data from the third data format to a fourth data format, wherein the third data format is a same as the second data format, and wherein the first data format is a same as the fourth data format.
18 . The system of claim 17 , wherein the first data format and the second data format is one of floating point (FP) 32, FP16, integer (INT) 8, unsigned int (UINT) 8, FP8, Brain FP (BF) 16, Fixed Point (FXP), In-phase Quadrature FP (IQFP), Quadrature (Q) format, quantization/dequantization, and scaling.
19 . The system of claim 17 further comprising a means for storing the converted data in the second format within a memory component of the ML hardware.