IP Library › Granted Patent US 12,505,002
Granted Patent B2
US 12,505,002 · App. 18/389,893 · Granted Dec 23, 2025

Single-step collective operations

Inventor: Richard Graham (Knoxville, TN)
Assignee: Mellanox Technologies, Ltd
G06F9/546
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,505,002
App. No.
18/389,893
Filed
Dec 20, 2023
Granted
Dec 23, 2025
Kind
B2
Examiner
LEE, BRYAN Y
Art Unit
2445
USPC
709/203
Abstract

A method for collective communications includes invoking a collective operation over a group of computing processes in which the processes concurrently transmit and receive data to and from other processes in the group via a communication medium. Messages are composed for transmission by source processes including metadata indicating how the data to be transmitted by the source processes in the collective operation are to be handled by destination processes that are to receive the data and also including in at least some of the messages the data to be transmitted by one or more of the source processes to one or more of the destination processes. The composed messages are transmitted concurrently from the source processes to the destination processes in the group over the communication medium. The data are processed by the destination processes in response to the metadata included in the messages received by the destination processes.

Claims (33)

1 . A method for collective communications, comprising:

invoking a collective operation over a group of computing processes in which the processes in the group concurrently transmit and receive data to and from other processes in the group via a communication medium;

composing, for transmission by source processes in the group, messages comprising metadata indicating how the data to be transmitted by the source processes in the collective operation are to be handled by destination processes that are to receive the data and also comprising in some of the messages the data to be transmitted by one or more of the source processes to one or more of the destination processes, while others of the messages comprise only the metadata;

transmitting the composed messages concurrently from the source processes to the destination processes in the group over the communication medium; and

processing the data by the destination processes in response to the metadata comprised in the messages received by the destination processes.

2 . The method according to claim 1 , wherein transmitting the composed messages comprises, for at least some of the messages:

transmitting the messages having respective sizes that are greater than a predefined threshold to the destination processes without aggregation; and

aggregating the messages for which the respective sizes are less than the predefined threshold and transmitting the aggregated messages to the destination processes.

3 . The method according to claim 1 , wherein invoking the collective operation comprises initiating an all-to-all-v, all-to-all-w, all-gather-v, gather-v, or scatter-v operation.

4 . A method for collective communications, comprising:

invoking a collective operation over a group of computing processes in which the processes in the group concurrently transmit and receive data to and from other processes in the group via a communication medium;

composing, for transmission by source processes in the group, messages comprising metadata indicating how the data to be transmitted by the source processes in the collective operation are to be handled by destination processes that are to receive the data and also comprising in at least some of the messages the data to be transmitted by one or more of the source processes to one or more of the destination processes;

transmitting the composed messages concurrently from the source processes to the destination processes in the group over the communication medium; and

processing the data by the destination processes in response to the metadata comprised in the messages received by the destination processes,

wherein when a given message contains the metadata without the data, the destination process retrieves the data from the source process using the metadata in the given message.

5 . The method according to claim 4 , wherein processing the data comprises transmitting, responsively to the metadata in a message received by a destination process, a request from the destination process to the source process to convey the data from the source process to the destination process, whereby the source process transmits the data to the destination process responsively to the request.

6 . The method according to claim 5 , wherein transmitting the request comprises transmitting a remote direct memory access (RDMA) read request from the destination process to the source process.

7 . A system for collective communications, comprising multiple processors, which are interconnected by a communication medium and are programmed to run respective computing processes such that upon receiving an invocation of a collective operation over a group of the processes in which the processes in the group are to concurrently transmit and receive data to and from other processes in the group via a communication medium,

the processors compose, for transmission by source processes in the group, messages comprising metadata indicating how the data to be transmitted by the source processes in the collective operation are to be handled by destination processes that are to receive the data and also comprising in some of the messages the data to be transmitted by one or more of the source processes to one or more of the destination processes, while others of the messages comprise only the metadata, and

the processors transmit the composed messages concurrently from the source processes to the destination processes in the group over the communication medium, and process the data in the destination processes in response to the metadata comprised in the messages received by the destination processes.

8 . The system according to claim 7 , wherein for at least some of the messages, the processors are to transmit the messages having respective sizes that are greater than a predefined threshold to the destination processes without aggregation, and the processors are to aggregate the messages for which the respective sizes are less than the predefined threshold and transmit the aggregated messages to the destination processes.

9 . The system according to claim 7 , wherein when a given message contains the metadata without the data, the destination process retrieves the data from the source process using the metadata in the given message.

10 . The system according to claim 9 , wherein the processors are to transmit, responsively to the metadata in a message received by a destination process, a request from the destination process to the source process to convey the data from the source process to the destination process, whereby the source process transmits the data to the destination process responsively to the request.

11 . The system according to claim 10 , wherein the request comprises a remote direct memory access (RDMA) read request transmitted from the destination process to the source process.

12 . The system according to claim 7 , wherein the collective operation comprises an all-to-all-v, all-to-all-w, all-gather-v, gather-v, or scatter-v operation.

13 . A computer software product for collective communications among a group of computing processes running on processors, which are interconnected by a communication medium, the product comprising a tangible, non-transitory computer-readable medium in which program instructions are stored, which instructions cause the processors, upon receiving an invocation of a collective operation over a group of the processes in which the processes in the group are to concurrently transmit and receive data to and from other processes in the group via a communication medium:

to compose, for transmission by source processes in the group, messages comprising metadata indicating how the data to be transmitted by the source processes in the collective operation are to be handled by destination processes that are to receive the data and also comprising in at least some of the messages the data to be transmitted by one or more of the source processes to one or more of the destination processes, while others of the messages comprise only the metadata, and

to transmit the composed messages concurrently from the source processes to the destination processes in the group over the communication medium, and to process the data in the destination processes in response to the metadata comprised in the messages received by the destination processes.

14 . The product according to claim 13 , wherein for at least some of the messages, the instructions cause the processors to transmit the messages having respective sizes that are greater than a predefined threshold to the destination processes without aggregation, and to aggregate the messages for which the respective sizes are less than the predefined threshold and transmit the aggregated messages to the destination processes.

15 . The product according to claim 13 , wherein when a given message contains the metadata without the data, the destination process retrieves the data from the source process using the metadata in the given message.

16 . The product according to claim 15 , wherein the instructions cause the processors to transmit, responsively to the metadata in a message received by a destination process, a request from the destination process to the source process to convey the data from the source process to the destination process, whereby the source process transmits the data to the destination process responsively to the request.

17 . The product according to claim 16 , wherein the request comprises a remote direct memory access (RDMA) read request transmitted from the destination process to the source process.

18 . The product according to claim 13 , wherein the collective operation comprises an all-to-all-v, all-to-all-w, all-gather-v, gather-v, or scatter-v operation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 20, 2023
From: GRAHAM, RICHARD
To: MELLANOX TECHNOLOGIES, LTD.
Reel/Frame 066090/0367 →
Continuity (3)
Continuation 18105846 · Feb 5, 2023
Provisional Application 63405505 · Sep 12, 2022
Related Publication 20240118949A1 · Apr 11, 2024
References Cited (383)
US 4933969A · Marshall et al. · 1990 [cited by applicant]
US 5068877A · Near et al. · 1991 [cited by applicant]
US 5325500A · Bell et al. · 1994 [cited by applicant]
US 5353412A · Douglas et al. · 1994 [cited by applicant]
US 5404565A · Gould et al. · 1995 [cited by applicant]
US 5408469A · Opher et al. · 1995 [cited by applicant]
US 5606703A · Brady et al. · 1997 [cited by applicant]
US 5944779A · Blum · 1999 [cited by applicant]
US 6041049A · Brady · 2000 [cited by applicant]
US 6115394A · Balachandran et al. · 2000 [cited by applicant]
US 6212197B1 · Christensen et al. · 2001 [cited by applicant]
US 6370502B1 · Wu et al. · 2002 [cited by applicant]
US 6438137B1 · Turner et al. · 2002 [cited by applicant]
US 6483804B1 · Muller et al. · 2002 [cited by applicant]
US 6507562B1 · Kadansky et al. · 2003 [cited by applicant]
US 6728862B1 · Wilson · 2004 [cited by applicant]
US 6816492B1 · Turner et al. · 2004 [cited by applicant]
US 6857004B1 · Howard et al. · 2005 [cited by applicant]
US 6937576B1 · Di Benedetto et al. · 2005 [cited by applicant]
US 7102998B1 · Golestani · 2006 [cited by applicant]
US 7124180B1 · Ranous · 2006 [cited by applicant]
US 7164422B1 · Wholey, III et al. · 2007 [cited by applicant]
US 7171484B1 · Krause et al. · 2007 [cited by applicant]
US 7224669B2 · Kagan et al. · 2007 [cited by applicant]
US 7245627B2 · Goldenberg et al. · 2007 [cited by applicant]
US 7313582B2 · Bhanot et al. · 2007 [cited by applicant]
US 7327693B1 · Rivers et al. · 2008 [cited by applicant]
US 7336646B2 · Muller · 2008 [cited by applicant]
US 7346698B2 · Hannaway · 2008 [cited by applicant]
US 7555549B1 · Campbell et al. · 2009 [cited by applicant]
US 7613774B1 · Caronni et al. · 2009 [cited by applicant]
US 7636424B1 · Halikhedkar et al. · 2009 [cited by applicant]
US 7636699B2 · Stanfill · 2009 [cited by applicant]
US 7676597B2 · Kagan et al. · 2010 [cited by applicant]
US 7738443B2 · Kumar · 2010 [cited by applicant]
US 7760743B2 · Shokri et al. · 2010 [cited by applicant]
US 8213315B2 · Crupnicoff et al. · 2012 [cited by applicant]
US 8255475B2 · Kagan et al. · 2012 [cited by applicant]
US 8370675B2 · Kagan · 2013 [cited by applicant]
US 8380880B2 · Gulley et al. · 2013 [cited by applicant]
US 8510366B1 · Anderson et al. · 2013 [cited by applicant]
US 8645663B2 · Kagan et al. · 2014 [cited by applicant]
US 8738891B1 · Karandikar et al. · 2014 [cited by applicant]
US 8761189B2 · Shachar et al. · 2014 [cited by applicant]
US 8768898B1 · Trimmer et al. · 2014 [cited by applicant]
US 8775698B2 · Archer et al. · 2014 [cited by applicant]
US 8811417B2 · Bloch et al. · 2014 [cited by applicant]
US 9110860B2 · Shahar · 2015 [cited by applicant]
US 9189447B2 · Faraj · 2015 [cited by applicant]
US 9294551B1 · Froese et al. · 2016 [cited by applicant]
US 9344490B2 · Bloch et al. · 2016 [cited by applicant]
US 9397960B2 · Arad et al. · 2016 [cited by applicant]
US 9456060B2 · Pope et al. · 2016 [cited by applicant]
US 9563426B1 · Bent et al. · 2017 [cited by applicant]
US 9626329B2 · Howard · 2017 [cited by applicant]
US 9756154B1 · Jiang · 2017 [cited by applicant]
US 10015106B1 · Florissi et al. · 2018 [cited by applicant]
US 10027601B2 · Narkis et al. · 2018 [cited by applicant]
US 10158702B2 · Bloch et al. · 2018 [cited by applicant]
US 10187400B1 · Castro et al. · 2019 [cited by applicant]
US 10284383B2 · Bloch et al. · 2019 [cited by applicant]
US 10296351B1 · Kohn et al. · 2019 [cited by applicant]
US 10305980B1 · Gonzalez et al. · 2019 [cited by applicant]
US 10318306B1 · Kohn et al. · 2019 [cited by applicant]
US 10320508B2 · Shimizu et al. · 2019 [cited by applicant]
US 10425350B1 · Florissi · 2019 [cited by applicant]
US 10515045B1 · Mattina · 2019 [cited by applicant]
US 10521283B2 · Shuler et al. · 2019 [cited by applicant]
US 10528518B2 · Graham et al. · 2020 [cited by applicant]
US 10541938B1 · Timmerman et al. · 2020 [cited by applicant]
US 10547553B2 · Shattah et al. · 2020 [cited by applicant]
US 10621489B2 · Appuswamy et al. · 2020 [cited by applicant]
US 10727966B1 · Izenberg et al. · 2020 [cited by applicant]
US 11088971B2 · Brody et al. · 2021 [cited by applicant]
US 11196586B2 · Graham et al. · 2021 [cited by applicant]
US 11336383B2 · Mula et al. · 2022 [cited by applicant]
US 11388263B2 · Levi et al. · 2022 [cited by applicant]
US 11609934B2 · Bensberg et al. · 2023 [cited by applicant]
US 11750699B2 · Graham et al. · 2023 [cited by applicant]
US 11922237B1 · Graham · 2024 [cited by applicant]
US 20020010844A1 · Noel et al. · 2002 [cited by applicant]
US 20020035625A1 · Tanaka · 2002 [cited by applicant]
US 20020150094A1 · Cheng et al. · 2002 [cited by applicant]
US 20020150106A1 · Kagan et al. · 2002 [cited by applicant]
US 20020152315A1 · Kagan et al. · 2002 [cited by applicant]
US 20020152327A1 · Kagan et al. · 2002 [cited by applicant]
US 20020152328A1 · Kagan et al. · 2002 [cited by applicant]
US 20020165897A1 · Kagan et al. · 2002 [cited by applicant]
US 20030002483A1 · Zwack · 2003 [cited by applicant]
US 20030018828A1 · Craddock et al. · 2003 [cited by applicant]
US 20030061417A1 · Craddock et al. · 2003 [cited by applicant]
US 20030065856A1 · Kagan et al. · 2003 [cited by applicant]
US 20030120835A1 · Kale et al. · 2003 [cited by applicant]
US 20040030745A1 · Boucher et al. · 2004 [cited by applicant]
US 20040062258A1 · Grow et al. · 2004 [cited by applicant]
US 20040078493A1 · Blumrich et al. · 2004 [cited by applicant]
US 20040120331A1 · Rhine et al. · 2004 [cited by applicant]
US 20040123071A1 · Stefan et al. · 2004 [cited by applicant]
US 20040174820A1 · Ricciulli · 2004 [cited by applicant]
US 20040252685A1 · Kagan et al. · 2004 [cited by applicant]
US 20040260683A1 · Chan et al. · 2004 [cited by applicant]
US 20050097300A1 · Gildea et al. · 2005 [cited by applicant]
US 20050122329A1 · Janus · 2005 [cited by applicant]
US 20050129039A1 · Biran et al. · 2005 [cited by applicant]
US 20050131865A1 · Jones et al. · 2005 [cited by applicant]
US 20050223118A1 · Tucker et al. · 2005 [cited by applicant]
US 20050281287A1 · Ninomi et al. · 2005 [cited by applicant]
US 20060095610A1 · Arndt et al. · 2006 [cited by applicant]
US 20060282838A1 · Gupta et al. · 2006 [cited by applicant]
US 20070127396A1 · Jain et al. · 2007 [cited by applicant]
US 20070127525A1 · Sarangam et al. · 2007 [cited by applicant]
US 20070162236A1 · Lamblin et al. · 2007 [cited by applicant]
US 20080040792A1 · Larson et al. · 2008 [cited by applicant]
US 20080059499A1 · Parkinson et al. · 2008 [cited by applicant]
US 20080104218A1 · Liang et al. · 2008 [cited by applicant]
US 20080126564A1 · Wilkinson · 2008 [cited by applicant]
US 20080168471A1 · Benner et al. · 2008 [cited by applicant]
US 20080181260A1 · Vonog et al. · 2008 [cited by applicant]
US 20080192750A1 · Ko et al. · 2008 [cited by applicant]
US 20080219159A1 · Chateau et al. · 2008 [cited by applicant]
US 20080244220A1 · Lin et al. · 2008 [cited by applicant]
US 20080263329A1 · Archer et al. · 2008 [cited by applicant]
US 20080288949A1 · Bohra et al. · 2008 [cited by applicant]
US 20080298380A1 · Rittmeyer et al. · 2008 [cited by applicant]
US 20080307082A1 · Cai et al. · 2008 [cited by applicant]
US 20090037377A1 · Archer et al. · 2009 [cited by applicant]
US 20090063816A1 · Arimilli et al. · 2009 [cited by applicant]
US 20090063817A1 · Arimilli et al. · 2009 [cited by applicant]
US 20090063891A1 · Arimilli et al. · 2009 [cited by applicant]
US 20090182814A1 · Tapolcai et al. · 2009 [cited by applicant]
US 20090240838A1 · Berg et al. · 2009 [cited by applicant]
US 20090247241A1 · Gollnick et al. · 2009 [cited by applicant]
US 20090292905A1 · Faraj · 2009 [cited by applicant]
US 20090296699A1 · Hefty · 2009 [cited by applicant]
US 20090327444A1 · Archer et al. · 2009 [cited by applicant]
US 20100017420A1 · Archer et al. · 2010 [cited by applicant]
US 20100049836A1 · Kramer · 2010 [cited by applicant]
US 20100074098A1 · Zeng et al. · 2010 [cited by applicant]
US 20100095086A1 · Eichenberger et al. · 2010 [cited by applicant]
US 20100185719A1 · Howard · 2010 [cited by applicant]
US 20100241828A1 · Yu et al. · 2010 [cited by applicant]
US 20100274876A1 · Kagan et al. · 2010 [cited by applicant]
US 20100329275A1 · Johnsen et al. · 2010 [cited by applicant]
US 20110060891A1 · Jia · 2011 [cited by applicant]
US 20110066649A1 · Berlyant et al. · 2011 [cited by applicant]
US 20110093258A1 · Xu et al. · 2011 [cited by applicant]
US 20110119673A1 · Bloch et al. · 2011 [cited by applicant]
US 20110173413A1 · Chen et al. · 2011 [cited by applicant]
US 20110219208A1 · Asaad · 2011 [cited by applicant]
US 20110238956A1 · Arimilli et al. · 2011 [cited by applicant]
US 20110258245A1 · Blocksome et al. · 2011 [cited by applicant]
US 20110276789A1 · Chambers et al. · 2011 [cited by applicant]
US 20120063436A1 · Thubert et al. · 2012 [cited by applicant]
US 20120117331A1 · Krause et al. · 2012 [cited by applicant]
US 20120131309A1 · Johnson · 2012 [cited by applicant]
US 20120254110A1 · Takemoto · 2012 [cited by applicant]
US 20120254344A1 · Archer · 2012 [cited by examiner]
US 20130117548A1 · Grover et al. · 2013 [cited by applicant]
US 20130159410A1 · Lee et al. · 2013 [cited by applicant]
US 20130159568A1 · Shahar et al. · 2013 [cited by applicant]
US 20130215904A1 · Zhou et al. · 2013 [cited by applicant]
US 20130250756A1 · Johri et al. · 2013 [cited by applicant]
US 20130312011A1 · Kumar et al. · 2013 [cited by applicant]
US 20130318525A1 · Palanisamy et al. · 2013 [cited by applicant]
US 20130336292A1 · Kore et al. · 2013 [cited by applicant]
US 20140019574A1 · Cardona et al. · 2014 [cited by applicant]
US 20140033217A1 · Vajda et al. · 2014 [cited by applicant]
US 20140040542A1 · Kim et al. · 2014 [cited by applicant]
US 20140047341A1 · Breternitz et al. · 2014 [cited by applicant]
US 20140095779A1 · Forsyth et al. · 2014 [cited by applicant]
US 20140122831A1 · Uliel et al. · 2014 [cited by applicant]
US 20140136811A1 · Fleischer et al. · 2014 [cited by applicant]
US 20140189308A1 · Hughes et al. · 2014 [cited by applicant]
US 20140211804A1 · Makikeni et al. · 2014 [cited by applicant]
US 20140258438A1 · Ayoub · 2014 [cited by applicant]
US 20140280420A1 · Khan · 2014 [cited by applicant]
US 20140281370A1 · Khan · 2014 [cited by applicant]
US 20140282613A1 · Jea et al. · 2014 [cited by applicant]
US 20140362692A1 · Wu et al. · 2014 [cited by applicant]
US 20140365548A1 · Mortensen · 2014 [cited by applicant]
US 20140379714A1 · Hankins · 2014 [cited by applicant]
US 20150046741A1 · Yen et al. · 2015 [cited by applicant]
US 20150055508A1 · Ashida et al. · 2015 [cited by applicant]
US 20150074373A1 · Sperber et al. · 2015 [cited by applicant]
US 20150106578A1 · Warfield et al. · 2015 [cited by applicant]
US 20150143076A1 · Khan · 2015 [cited by applicant]
US 20150143077A1 · Khan · 2015 [cited by applicant]
US 20150143078A1 · Khan et al. · 2015 [cited by applicant]
US 20150143079A1 · Khan · 2015 [cited by applicant]
US 20150143085A1 · Khan · 2015 [cited by applicant]
US 20150143086A1 · Khan · 2015 [cited by applicant]
US 20150154058A1 · Miwa et al. · 2015 [cited by applicant]
US 20150178211A1 · Hiramoto et al. · 2015 [cited by applicant]
US 20150180785A1 · Annamraju · 2015 [cited by applicant]
US 20150188987A1 · Reed et al. · 2015 [cited by applicant]
US 20150193271A1 · Archer et al. · 2015 [cited by applicant]
US 20150212972A1 · Boettcher et al. · 2015 [cited by applicant]
US 20150261720A1 · Kagan et al. · 2015 [cited by applicant]
US 20150269116A1 · Raikin et al. · 2015 [cited by applicant]
US 20150278347A1 · Meyer et al. · 2015 [cited by applicant]
US 20150318015A1 · Bose et al. · 2015 [cited by applicant]
US 20150347012A1 · Dewitt et al. · 2015 [cited by applicant]
US 20150365494A1 · Cardona et al. · 2015 [cited by applicant]
US 20150379022A1 · Puig et al. · 2015 [cited by applicant]
US 20160055225A1 · Xu et al. · 2016 [cited by applicant]
US 20160092362A1 · Barron et al. · 2016 [cited by applicant]
US 20160105494A1 · Reed et al. · 2016 [cited by applicant]
US 20160112531A1 · Milton et al. · 2016 [cited by applicant]
US 20160117277A1 · Raindel et al. · 2016 [cited by applicant]
US 20160119244A1 · Wang et al. · 2016 [cited by applicant]
US 20160179537A1 · Kunzman et al. · 2016 [cited by applicant]
US 20160219009A1 · French · 2016 [cited by applicant]
US 20160246646A1 · Craciunas et al. · 2016 [cited by applicant]
US 20160248656A1 · Anand et al. · 2016 [cited by applicant]
US 20160283422A1 · Crupnicoff et al. · 2016 [cited by applicant]
US 20160294793A1 · Larson et al. · 2016 [cited by applicant]
US 20160299872A1 · Vaidyanathan et al. · 2016 [cited by applicant]
US 20160342568A1 · Burchard et al. · 2016 [cited by applicant]
US 20160352598A1 · Reinhardt et al. · 2016 [cited by applicant]
US 20160364350A1 · Sanghi et al. · 2016 [cited by applicant]
US 20170063613A1 · Bloch et al. · 2017 [cited by applicant]
US 20170093715A1 · McGhee et al. · 2017 [cited by applicant]
US 20170116154A1 · Palmer et al. · 2017 [cited by applicant]
US 20170187496A1 · Shalev et al. · 2017 [cited by applicant]
US 20170187589A1 · Pope et al. · 2017 [cited by applicant]
US 20170187629A1 · Shalev et al. · 2017 [cited by applicant]
US 20170187846A1 · Shalev et al. · 2017 [cited by applicant]
US 20170192782A1 · Valentine et al. · 2017 [cited by applicant]
US 20170199844A1 · Burchard et al. · 2017 [cited by applicant]
US 20170262517A1 · Horowitz et al. · 2017 [cited by applicant]
US 20170308329A1 · A et al. · 2017 [cited by applicant]
US 20170331926A1 · Raveh et al. · 2017 [cited by applicant]
US 20170344589A1 · Kafai et al. · 2017 [cited by applicant]
US 20180004530A1 · Vorbach · 2018 [cited by applicant]
US 20180046901A1 · Xie et al. · 2018 [cited by applicant]
US 20180047099A1 · Bonig et al. · 2018 [cited by applicant]
US 20180089278A1 · Bhattacharjee et al. · 2018 [cited by applicant]
US 20180091442A1 · Chen et al. · 2018 [cited by applicant]
US 20180097721A1 · Matsui et al. · 2018 [cited by applicant]
US 20180115529A1 · Munger et al. · 2018 [cited by applicant]
US 20180173673A1 · Daglis et al. · 2018 [cited by applicant]
US 20180262551A1 · Demeyer et al. · 2018 [cited by applicant]
US 20180278549A1 · Mula et al. · 2018 [cited by applicant]
US 20180285151A1 · Wang et al. · 2018 [cited by applicant]
US 20180285316A1 · Thorson et al. · 2018 [cited by applicant]
US 20180287725A1 · Rabinovich et al. · 2018 [cited by applicant]
US 20180287928A1 · Levi et al. · 2018 [cited by applicant]
US 20180302324A1 · Kasuya · 2018 [cited by applicant]
US 20180321912A1 · Li et al. · 2018 [cited by applicant]
US 20180321938A1 · Boswell et al. · 2018 [cited by applicant]
US 20180349212A1 · Liu et al. · 2018 [cited by applicant]
US 20180367465A1 · Levi · 2018 [cited by applicant]
US 20180375781A1 · Chen et al. · 2018 [cited by applicant]
US 20190018805A1 · Benisty · 2019 [cited by applicant]
US 20190026250A1 · Das Sarma et al. · 2019 [cited by applicant]
US 20190044827A1 · Ganapathi · 2019 [cited by examiner]
US 20190044875A1 · Murty et al. · 2019 [cited by applicant]
US 20190044889A1 · Serres et al. · 2019 [cited by applicant]
US 20190056972A1 · Zhou et al. · 2019 [cited by applicant]
US 20190065208A1 · Liu et al. · 2019 [cited by applicant]
US 20190068501A1 · Schneder et al. · 2019 [cited by applicant]
US 20190102090A1 · Guim et al. · 2019 [cited by applicant]
US 20190102179A1 · Fleming et al. · 2019 [cited by applicant]
US 20190102338A1 · Tang et al. · 2019 [cited by applicant]
US 20190102640A1 · Balasubramanian · 2019 [cited by applicant]
US 20190114533A1 · Ng et al. · 2019 [cited by applicant]
US 20190121388A1 · Knowles et al. · 2019 [cited by applicant]
US 20190124524A1 · Gormley · 2019 [cited by applicant]
US 20190138638A1 · Pal et al. · 2019 [cited by applicant]
US 20190141133A1 · Rajan et al. · 2019 [cited by applicant]
US 20190147092A1 · Pal et al. · 2019 [cited by applicant]
US 20190149486A1 · Bohrer et al. · 2019 [cited by applicant]
US 20190149488A1 · Bansal et al. · 2019 [cited by applicant]
US 20190171612A1 · Shahar et al. · 2019 [cited by applicant]
US 20190187774A1 · Yi et al. · 2019 [cited by applicant]
US 20190235866A1 · Das Sarma et al. · 2019 [cited by applicant]
US 20190278737A1 · Kozomora et al. · 2019 [cited by applicant]
US 20190303168A1 · Fleming, Jr. et al. · 2019 [cited by applicant]
US 20190303263A1 · Fleming, Jr. et al. · 2019 [cited by applicant]
US 20190319730A1 · Webb et al. · 2019 [cited by applicant]
US 20190324431A1 · Cella et al. · 2019 [cited by applicant]
US 20190339688A1 · Cella et al. · 2019 [cited by applicant]
US 20190347099A1 · Eapen et al. · 2019 [cited by applicant]
US 20190369994A1 · Parandeh Afshar et al. · 2019 [cited by applicant]
US 20190377580A1 · Vorbach · 2019 [cited by applicant]
US 20190379714A1 · Levi et al. · 2019 [cited by applicant]
US 20200005859A1 · Chen et al. · 2020 [cited by applicant]
US 20200034145A1 · Bainville et al. · 2020 [cited by applicant]
US 20200057748A1 · Danilak · 2020 [cited by applicant]
US 20200103894A1 · Cella et al. · 2020 [cited by applicant]
US 20200106828A1 · Elias et al. · 2020 [cited by applicant]
US 20200137013A1 · Jin et al. · 2020 [cited by applicant]
US 20200202246A1 · Lin et al. · 2020 [cited by applicant]
US 20200265043A1 · Graham et al. · 2020 [cited by applicant]
US 20200274733A1 · Graham et al. · 2020 [cited by applicant]
US 20210203621A1 · Ylisirnio et al. · 2021 [cited by applicant]
US 20210218808A1 · Graham et al. · 2021 [cited by applicant]
US 20210234753A1 · Ben-Moshe et al. · 2021 [cited by applicant]
US 20210243140A1 · Levi et al. · 2021 [cited by applicant]
US 20210297151A1 · Levi et al. · 2021 [cited by applicant]
US 20220006606A1 · Levi et al. · 2022 [cited by applicant]
US 20220029854A1 · Graham et al. · 2022 [cited by applicant]
US 20220078043A1 · Marcovitch et al. · 2022 [cited by applicant]
US 20220188147A1 · Nudelman et al. · 2022 [cited by applicant]
US 20220201103A1 · Keppel et al. · 2022 [cited by applicant]
US 20230089099A1 · Nudelman et al. · 2023 [cited by applicant]
US 20230292149A1 · Wu et al. · 2023 [cited by applicant]
US 20230401116A1 · Hans et al. · 2023 [cited by applicant]
US 20240305685A1 · Chalakov · 2024 [cited by examiner]
US 20250193310A1 · Xie et al. · 2025 [cited by applicant]
AU 2012216611B2 · 2013 [cited by applicant]
CN 103095607A · 2013 [cited by applicant]
CN 109617640A · 2019 [cited by applicant]
WO 03044677A1 · 2003 [cited by applicant]
WO 2016150833A1 · 2016 [cited by applicant]
WO 2016167915A1 · 2016 [cited by applicant]
IEEE Standard 1588-2008, “IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems”, pp. 1-289, year 2008. [cited by applicant]
Weibel et al., “Implementation and Performance of Time Stamping Techniques”, 2004 Conference on IEEE 1588, pp. 1-29, Sep. 28, 2004. [cited by applicant]
InfiniBandTM Architecture Specification vol. 1, Release 1.2.1,pp. 1-1727, Nov. 2007. [cited by applicant]
Lu et al., “A Fast CRC Update Implementation”, Computer Engineering Laboratory, Electrical Engineering Department, pp. 113-120, Oct. 8, 2003. [cited by applicant]
Mellette et al., “Toward Optical Switching in the Data Center”, IEEE 19th International Conference on High Performance Switching and Routing (HPSR), pp. 1-6, Bucharest, Romania, Jun. 18-20, 2018. [cited by applicant]
Bakopoulos et al., “Nephele: an end-to-end scalable and dynamically reconfigurable optical architecture for application-aware SDN cloud datacenters”, IEEE Communications Magazine, vol. 56, issue 2, pp. 1-26, Feb. 2018. [cited by applicant]
O-RAN Alliance, “O-RAN Fronthaul Working Group; Control, User and Synchronization Plane Specification”, ORAN-WG4.CUS.0-v01.00, Technical Specification, pp. 1-189, year 2019. [cited by applicant]
Vattikonda et al., “Practical TDMA for Datacenter Ethernet”, EuroSys conference, Bern, Switzerland, pp. 225-238, Apr. 10-13, 2012. [cited by applicant]
Ericsson AB et al., “Common Public Radio Interface: eCPRI Interface Specification”, V2.0, pp. 1-109, May 10, 2019. [cited by applicant]
Xilinx Inc., “Radio over Ethernet Framer v2.1”, PB056 (v2.1), pp. 1-9, Oct. 30, 2019. [cited by applicant]
Weibel, H., “High Precision Clock Synchronization according to IEEE 1588 Implementation and Performance Issues”, Zurich University of Applied Sciences, pp. 1-9, Jan. 17, 2005. [cited by applicant]
Sanchez-Palencia, J., “[RFC,v3,net-next,00/18] Time based packet transmission”, pp. 1-14, Mar. 7, 2018. [cited by applicant]
IEEE Std 802.1Qaz™, “IEEE Standard for Local and metropolitan area networks-Media Access Control (MAC) Bridges and Virtual Bridged Local Area Networks-Amendment 18: Enhanced Transmission Selection for Bandwidth Sharing … [cited by applicant]
Crupnicoff et al., “Deploying Quality of Service and Congestion Control in InfiniBand-based Data Center Networks” White Paper, Mellanox Technologies Inc, Rev 1.0, pp. 1-19, Oct. 15, 2005. [cited by applicant]
Mathey et al., “Scalable Deadlock-Free Deterministic Minimal-Path Routing Engine for InfiniBand-Based Dragonfly Networks”, IEEE Transactions on Parallel and Distributed Systems, vol. 29, No. 1, pp. 183-197, Jan. 2018. [cited by applicant]
Wikipedia,Precision Time Protocol, pp. 1-9, Apr. 20, 2020. [cited by applicant]
SMPTE Standard, “Professional Media Over Managed IP Networks: Traffic Shaping and Delivery Timing for Video”, The Society of Motion Picture and Television Engineers, pp. 1-17, Nov. 22, 2017. [cited by applicant]
Wikipedia, “Time-Sensitive Networking”, pp. 1-12, Mar. 5, 2020. [cited by applicant]
Wikipedia, “Memory Protection,” pp. 1-6, last edited May 23, 2021. [cited by applicant]
Mills, “Network Time Protocol (NTP),” RFC 0958, pp. 2-15, Sep. 1985. [cited by applicant]
Mills, “Network Time Protocol (Version 1): Specification and Implementation,” RFC 1059, pp. 2-59, Jul. 1988. [cited by applicant]
Mills, “Internet Time Synchronization: The Network Time Protocol,” IEEE Transactions on Communication, vol. 39, No. 10, pp. 1482-1493, Oct. 1991. [cited by applicant]
Mills, “Network Time Protocol (Version 3): Specification, Implementation and Analysis,” RFC 1305, pp. 1-96, Mar. 1992. [cited by applicant]
Xu et al., “SLOAVx: Scalable Logarithmic AlltoallV Algorithm for Hierarchical Multicore Systems”, 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, pp. 369-376, year 2013. [cited by applicant]
NVIDIA Corporation, “NVIDIA Scalable Hierarchical Aggregation and Reduction Protocol (SHARP),” Rev. 3.0.0, pp. 1-3, year 2023. [cited by applicant]
Graham, U.S. Appl. No. 18/074,563, filed Dec. 5, 2022. [cited by applicant]
Graham, U.S. Appl. No. 18/105,846, filed Feb. 5, 2023. [cited by applicant]
Graham, U.S. Appl. No. 63/356,923, filed Jun. 29, 2022. [cited by applicant]
Oltchik et al, U.S. Appl. No. 18/451,134, filed Aug. 17, 2023. [cited by applicant]
Graham et al., U.S. Appl. No. 18/513,565, filed Nov. 19, 2023. [cited by applicant]
Levi et al., U.S. Appl. No. 18/523,991, filed Nov. 30, 2023. [cited by applicant]
CN Application # 202110742521.4 Office Action dated Jan. 30, 2024. [cited by applicant]
Yang et al., “SwitchAgg: A Further Step Toward In-Network Computing,” 2019 IEEE International Conference on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Commun… [cited by applicant]
Pacheco, “A User's Guide to MPI,” Department of Mathematics, University of San Francisco, CA, USA, pp. 1-51, Mar. 30, 1998. [cited by applicant]
Wikipedia, “Message Passing Interface,” pp. 1-16, last edited Nov. 7, 2021, as downloaded from https://en.wikipedia.org/wiki/Message_Passing_Interface. [cited by applicant]
“Message Passing Interface (MPI): History and Evolution,” Virtual Workshop, Cornell University Center for Advanced Computing, NY, USA, pp. 1-2, year 2021, as downloaded from https://cvw.cac.cornell.edu/mpi/history. [cited by applicant]
Mellanox Technologies, “InfiniScale IV: 36-port 40GB/s Infiniband Switch Device”, pp. 1-2, year 2009. [cited by applicant]
Mellanox Technologies Inc., “Scaling 10Gb/s Clustering at Wire-Speed”, pp. 1-8, year 2006. [cited by applicant]
IEEE 802.1D Standard “IEEE Standard for Local and Metropolitan Area Networks—Media Access Control (MAC) Bridges”, IEEE Computer Society, pp. 1-281, Jun. 9, 2004. [cited by applicant]
IEEE 802.1AX Standard “IEEE Standard for Local and Metropolitan Area Networks—Link Aggregation”, IEEE Computer Society, pp. 1-163, Nov. 3, 2008. [cited by applicant]
Turner et al., “Multirate Clos Networks”, IEEE Communications Magazine, pp. 1-11, Oct. 2003. [cited by applicant]
Thayer School of Engineering, “An Slightly Edited Local Elements of Lectures 4 and 5”, Dartmouth College, pp. 1-5, Jan. 15, 1998 http://people.seas.harvard.edu/˜jones/cscie129/nu_lectures/lecture11/switching/clos_networ… [cited by applicant]
“MPI: A Message-Passing Interface Standard,” Message Passing Interface Forum, version 3.1, pp. 1-868, Jun. 4, 2015. [cited by applicant]
Coti et al., “MPI Applications on Grids: a Topology Aware Approach,” Proceedings of the 15th International European Conference on Parallel and Distributed Computing (EuroPar'09), pp. 1-12, Aug. 2009. [cited by applicant]
Petrini et al., “The Quadrics Network (QsNet): High-Performance Clustering Technology,” Proceedings of the 9th IEEE Symposium on Hot Interconnects (Hotl'01), pp. 1-6, Aug. 2001. [cited by applicant]
Sancho et al., “Efficient Offloading of Collective Communications in Large-Scale Systems,” Proceedings of the 2007 IEEE International Conference on Cluster Computing, pp. 1-10, Sep. 17-20, 2007. [cited by applicant]
Priest et al., “You've Got Mail (YGM): Building Missing Asynchronous Communication Primitives”, IEEE International Parallel and Distributed Processing Symposium Workshops, pp. 221-230, year 2019. [cited by applicant]
InfiniBand Architecture Specification, vol. 1, Release 1.2.1, pp. 1-1727, Nov. 2007. [cited by applicant]
Deming, “Infiniband Architectural Overview”, Storage Developer Conference, pp. 1-70, year 2013. [cited by applicant]
Fugger et al., “Reconciling fault-tolerant distributed computing and systems-on-chip”, Distributed Computing, vol. 24, Issue 6, pp. 323-355, Jan. 2012. [cited by applicant]
Wikipedia, “System on a chip”, pp. 1-4, Jul. 6, 2018. [cited by applicant]
Villavieja et al., “On-chip Distributed Shared Memory”, Computer Architecture Department, pp. 1-10, Feb. 3, 2011. [cited by applicant]
Wikipedia, “Nagle's algorithm”, pp. 1-4, Dec. 12, 2019. [cited by applicant]
Bruck et al., “Efficient Algorithms for All-to-All Communications in Multiport Message-Passing Systems”, IEEE Transactions on Parallel and Distributed Systems, vol. 8, No. 11, pp. 1143-1156, Nov. 1997. [cited by applicant]
Gainaru et al., “Using InfiniBand Hardware Gather-Scatter Capabilities to Optimize MPI All-to-All”, EuroMPI '16, Edinburgh, United Kingdom, pp. 1-13, year 2016. [cited by applicant]
Pjesivac-Grbovic et al., “Performance analysis of MPI collective operations”, Cluster Computing, pp. 1-25, year 2007. [cited by applicant]
Bruck et al., “Efficient Algorithms for All-to-All Communications in Multiport Message-Passing Systems”, Proceedings of the sixth annual ACM symposium on Parallel algorithms and architectures, pp. 298-309, Aug. 1, 1994. [cited by applicant]
Chiang et al., “Toward supporting data parallel programming on clusters of symmetric multiprocessors”, Proceedings International Conference on Parallel and Distributed Systems, pp. 607-614, Dec. 14, 1998. [cited by applicant]
Danalis et al., “PTG: an abstraction for unhindered parallelism”, 2014 Fourth International Workshop on Domain-Specific Languages and High-Level Frameworks for High Performance Computing, pp. 1-10, Nov. 17, 2014. [cited by applicant]
Cosnard et al., “Symbolic Scheduling of Parameterized Task Graphs on Parallel Machines,” Combinatorial Optimization book series (COOP, vol. 7), pp. 217-243, year 2000. [cited by applicant]
Jeannot et al., “Automatic Multithreaded Parallel Program Generation for Message Passing Multiprocessors using paramerized Task Graphs”, World Scientific, pp. 1-8, Jul. 23, 2001. [cited by applicant]
Stone, “An Efficient Parallel Algorithm for the Solution of a Tridiagonal Linear System of Equations,” Journal of the Association for Computing Machinery, vol. 10, No. 1, pp. 27-38, Jan. 1973. [cited by applicant]
Kogge et al., “A Parallel Algorithm for the Efficient Solution of a General Class of Recurrence Equations,” IEEE Transactions on Computers, vol. C-22, No. 8, pp. 786-793, Aug. 1973. [cited by applicant]
Hoefler et al., “Message Progression in Parallel Computing—To Thread or not to Thread?”, 2008 IEEE International Conference on Cluster Computing, pp. 1-10, Tsukuba, Japan, Sep. 29-Oct. 1, 2008. [cited by applicant]
Wikipedia, “Loop unrolling,” pp. 1-9, last edited Sep. 9, 2020 downloaded from https://en.wikipedia.org/wiki/Loop_unrolling. [cited by applicant]
Chapman et al., “Introducing OpenSHMEM: SHMEM for the PGAS Community,” Proceedings of the Forth Conferene on Partitioned Global Address Space Programming Model, pp. 1-4, Oct. 2010. [cited by applicant]
Non-Final Office Action # 18074563 dated Jul. 23, 2025. [cited by applicant]