System and method for scalable movement and replication of data
A method of multicast data transfer including accessing a source address to a source location of mapped memory which stores source data, accessing multiple destination addresses to corresponding destination locations of the mapped memory, and for each of at least one section of the source data, reading the section using the source address, storing the section into a local memory of a data transfer device, and writing the section from the local memory to each destination location in the mapped memory using the destination addresses. Separate source and destination attributes may be provided, so that the source and each destination may have different attributes for reading and storing data. The source and each destination may have any number of data buffers accessible by corresponding links provided in data structures supporting the data transfer. The source data may be divided into sections and handled section by section.
1. A method of multicast data transfer, comprising:
accessing a source address to a source location of mapped memory which stores source data;
accessing a plurality of destination addresses to a corresponding plurality of destination locations of the mapped memory;
for each of a plurality of sections of the source data, one section at a time:
reading a section using the source address;
storing the section into a local memory of a data transfer device; and
writing the section from the local memory to each of the plurality of destination locations in the mapped memory using the plurality of destination addresses.
2. The method of claim 1 , wherein said accessing a plurality of destination addresses comprises accessing a plurality of destination records each storing a corresponding one of the plurality of destination addresses and each storing at least one destination attribute, and wherein said writing the section from the local memory to each of the plurality of destination locations comprises using the at least one destination attribute stored in a corresponding destination record.
3. The method of claim 1 , further comprising retrieving a plurality of unique destination attributes, each used during said writing the section from the local memory to a corresponding one of the plurality of destination locations.
4. The method of claim 1 , wherein after writing a section of the source data from the local memory to the plurality of destination locations in the mapped memory, further comprising updating a corresponding one of a plurality of break points within the local memory to point to a next memory location of a corresponding destination location for writing a subsequent section of the source data.
5. The method of claim 1 , wherein said accessing each of a plurality of destination addresses comprises accessing a plurality of links in the mapped memory used for accessing a corresponding plurality of destination data buffers in the mapped memory.
6. The method of claim 1 , wherein said accessing each of a plurality of destination addresses comprises accessing a corresponding one of a plurality of destination descriptors in the mapped memory, wherein each destination descriptor includes at least one destination attribute.
7. The method of claim 1 , further comprising:
retrieving a completion status enable value in the mapped memory indicating whether to report completion to a source device and indicating whether to report completion to any of a plurality of destination devices; and
retrieving a plurality of destination completion status enable values, each indicating whether to report completion to a corresponding one of the plurality of destination devices.
8. A method of transferring data in mapped memory, comprising:
generating a source record to include a source address for accessing source data stored at a source location in the mapped memory and wherein the source data is divided into a plurality of sections;
generating a plurality of destination records, each destination record including a corresponding one of a plurality of destination addresses for accessing a corresponding one of a plurality of destination locations in the mapped memory; and
performing a multicast transfer operation for the plurality of sections of the source data, one section at a time, comprising:
reading a section of the source data from the source location using the source address;
storing the section into a local memory of a data transfer device;
writing the section stored in the local memory to each of the plurality of destination locations using the plurality of destination addresses; and
repeating said reading, storing and writing for each of the plurality of sections of the source data.
9. The method of claim 8 , wherein said generating a plurality of destination records includes storing at least one attribute in each destination record which is used for accessing a corresponding destination location.
10. The method of claim 8 , further comprising generating a plurality of destination descriptors, each including at least one attribute which is used for writing source data to a corresponding one of the plurality of destination locations.
11. The method of claim 8 , further comprising:
generating at least one destination data block in the mapped memory accessible via a corresponding one of the plurality of destination records;
providing a destination data buffer in the at least one destination data block; and
storing a destination offset in the corresponding one of the plurality of destination records used for accessing the destination data buffer.
12. The method of claim 8 , further comprising:
providing a plurality of destination data buffers in the mapped memory;
generating at least one destination data block accessible via a corresponding one of the plurality of destination records, wherein said at least one destination block includes a data table comprising a plurality of links each pointing to a corresponding one of the plurality of destination data buffers; and
storing a destination offset in a corresponding destination record in which the data table is accessible via a corresponding destination address and the destination offset.
13. The method of claim 8 , further comprising updating a corresponding one of a plurality of break points stored in the local memory after writing a section of the source data to one of the plurality of destination locations.
14. A data transfer system for performing a multicast operation, comprising:
a system interface;
a mapped memory comprising at least one memory device coupled to said system interface, wherein said mapped memory stores source data divided into a plurality of sections at a source location accessible using a source address and wherein said mapped memory includes a plurality of destination locations each accessible using a corresponding one of a plurality of destination addresses; and
a data transfer device coupled to said system interface, wherein said data transfer device comprises:
a local memory; and
a data transfer engine that is configured to transfer said source data, one section at a time, from said source location to each of said plurality of destination locations, wherein said data transfer engine uses said source address to read a next one of said plurality of sections of said source data and to store said next one of said plurality of sections of said source data into said local memory, and wherein said data transfer engine uses each of said plurality of destination addresses to write said next one of said plurality of sections from said local memory to each of said plurality of destination locations.
15. The data transfer system of claim 14 , wherein after writing each section, said data transfer engine stores a corresponding break point into said local memory for a corresponding one of said plurality of destination locations.
16. The data transfer system of claim 14 , wherein said mapped memory stores a list of destination records each storing a corresponding one of said destination addresses and each storing at least one attribute used for accessing a corresponding one of said plurality of destination locations.
17. The data transfer system of claim 14 , wherein each of said plurality of destination addresses is used to access a corresponding one of a plurality of destination descriptors stored in said mapped memory, and wherein each of said plurality of destination descriptors stores at least one destination attribute used for writing data to a corresponding destination location.
18. The data transfer system of claim 14 , further comprising a source device coupled to said system interface which stores said source data, which determines said plurality of destination locations in said mapped memory, and which invokes said data transfer engine to perform said multicast operation.
19. The data transfer system of claim 14 , further comprising:
at least one core processor coupled to said system interface; and
an input/output system coupled to said system interface comprising a plurality of input/output devices including said data transfer device.