Distributed architecture for data transparency platform

1 . A computer-implemented method of providing passive market data transparency using a distributed computing architecture, the method comprising:

a) generating a first database, wherein generating the first database comprises

i) aggregating passive market data from one or more sources, wherein the passive market data comprises index data for one or more indices, exchange traded fund (ETF) data for one or more ETFs, and asset data for one or more assets, where the aggregated passive market data comprises at least time-series data structured as one or more tensors with a plurality of indices, with a tensor represented as a multi-dimensional array data structure,

ii) converting the aggregated passive market data from a non-standard format into a standard format, wherein the standard format is based on a taxonomy of the one or more indices, the one or more ETFs, and the one or more assets across the one or more sources, and

iii) generating a standard identifier according to the taxonomy for each of the one or more indices, the one or more ETFs, and the one or more assets, wherein the standard identifier is linked to all identifiers used for the same asset, index, or ETF;

b) inputting the one or more tensors with the plurality of indices to a machine learning model to predict bidirectional pass through data between at least an asset and an ETF or between at least an index and an ETF in an ETF/index/asset network, with the machine learning model including a multi-dimensional representation of relational structure between the asset and the ETF or the index and the ETF as one or more bidirectional, time-series data structures, wherein the ETF/index/asset network comprises the relationships and associated non-linear interactions of one or more ETFs, one or more indices and one or more assets,

wherein the bidirectional passthrough data comprise one or more characteristics, events or flows, the machine learning model comprising:

a first class of nodes and a second class of nodes,

wherein one or more of the nodes in the first class represent at least first time-series data for a given bidirectional pass through data, with the first time-series data being ETF data,

wherein one or more of the nodes in the second class represent at least second time-series data for the given bidirectional pass through data, with the second time-series data being index data or asset data first directional edges connecting nodes in the first class to nodes in the second class, each first edge specifying a first weight representing a first dependency between a node in the first class and a node in the second class;

second directional edges connecting nodes in the second class to nodes in the first class, each second edge specifying a second weight representing a second dependency between a node in the first class and a node in the second class;

wherein a bidirectional structure of the one or more bidirectional, time-series data structures enables frame of reference translation between nodes in the first class and nodes in the second class;

outputting, from the machine learning model, vectorized computations of bidirectional dataflow between nodes in the first and second classes;

based on the output vectorized computations, training the machine learning model to predict with increased accuracy bidirectional pass through data between nodes in the first and second classes of the machine learning model;

wherein the vectorized computations comprise parallelization of computing the bidirectional dataflow over all indices of at least one of the one or more tensors; and

wherein the distributed computing architecture is configured to perform vectorization that handles dependence and recurrence between indices of the one or more tensors, including permitting a time indexed value associated with a given tensor index to depend on a value associated with a prior time index without explicit iteration over indices and without referring to a specific time-coordinate, by executing one or more sub-vectorized dependency management algorithms that perform concurrent calculation of dependent variable relationships;

c) applying, in the distributed computing architecture, the trained machine learning model over at least the time-series data to generate pass through data based at least in part on values of the bidirectional pass through data;

wherein applying the trained machine learning model comprises executing the one or more sub-vectorized dependency management algorithms to concurrently calculate dependent variable relationships in the time-series data without sequentially iterating over indices of the one or more tensors;

d) storing, in a storage device, the trained machine learning model;

e) generating a second database by aggregating the pass through data, and converting the pass through data to a standard format, wherein the standard format is based on a taxonomy of the one or more generated bidirectional, time-series data structures based at least in part on the bidirectional pass through data and their pass through effects, and generating a standard identifier according to the taxonomy,

wherein the second database is searchable by the standard identifiers;

f) providing an analytics layer for synthesizing the first database, the one or more generated bidirectional, time-series data structures including the bidirectional pass through data, the pass through data, and the second database, and producing an analytics output, wherein producing the analytics output comprises:

i) receiving a search query to search, the one or more generated bidirectional, time-series data structures including the bidirectional pass through data, the pass through data, or the second database, mapping the search query to one or more of the standard identifiers, and returning a set of pass through data across the one or more dimensions of the multi-dimensional ETF/index/asset network, for the one or more standard identifiers of the search query, and

ii) performing one or more tasks using the returned set of passive market data or the set of pass through data, wherein the one or more tasks comprise: mapping passive market activity, performing portfolio risk monitoring, performing direct indexing, performing compliance monitoring, performing conflict monitoring, tracking information flows, or a combination thereof; and

g) generating a passive market visual interface for navigating and interpreting the analytics outputs, wherein the passive market visual interface is configured to be presented on a display, and wherein the passive market visual interface comprises at least one of a market overview window, a forecast and recommendation window, an event and flows window, an ownership window, a portfolio performance, or a risk window.

2 . The method of claim 1 , wherein the passive market data comprises one or more of ETF data, ownership data, relationship data, or associated ETF service provider data, for the ETF/index/asset network.

3 . The method of claim 1 , wherein the first database returns passive market data for a searched asset across global index providers, ETFs, or Fund Issuers.

4 . The method of claim 1 , wherein the ETF data comprises one or more of data related to Authorized Participants (AP), market makers, creation/redemption data, AP buying/selling data, or a combination thereof.

5 . The method of claim 1 , wherein aggregating and standardizing passive market data further comprises aggregating and standardizing the passive market data from multiple reporting sources and formats.

6 . The method of claim 1 , wherein the second database returns pass through data for a searched asset across global index providers, ETFs, or Fund Issuers.

7 . The method of claim 1 , wherein the time series data comprises calendar time series data.

8 . The method of claim 1 , wherein the time series data comprises relative time series data.

9 . The method of claim 1 , wherein applying the first algorithm is automated.

10 . The method of claim 1 , wherein the first algorithm comprises a machine learning algorithm.

11 . The method of claim 1 , wherein the pass through flows comprise one or more of fund flows, flows of risk or derivative/swap exposure, flows of orders or changes to existing or latent order books, flows of consolidated trades and quotes, flows of distributions/dividends or equivalent yields, flows of payments, flows of dark pool or odd lot transactional information, flows of tax liability or tax related information, flows of proxy or shareholder votes, or flows of Material Non-Public Insider Information, or a combination thereof.

12 . The method of claim 1 , further comprising calculating the pass through data by applying one or more of an information flow, capital flow, or data flow to the one or more bidirectional, time-series data structures.

13 . The method of claim 1 , further comprising calculating the pass through data based at least in part on creation and redemption variables.

14 . The method of claim 1 , further comprising calculating the pass through data based at least in part on AP selling and buying variables.

15 . The method of claim 1 , wherein the pass through data comprises passing through an integer input, a function, or a combination thereof through the one or more bidirectional, time-series data structures.

16 . The method of claim 1 , further comprising generating a visual display by generating an actionable recommendation or making a prediction based at least in part on the calculated pass through data.

17 . The method of claim 1 , wherein producing the analytics output comprises applying a second algorithm to rank, sort, and/or quantify the influence of pass through variables across the ETF/index/asset network.

18 . The method of claim 17 , wherein the second algorithm comprises a machine learning algorithm.

19 . The method of claim 1 , wherein the first database and the second database are stored in a memory.

20 . The method of claim 1 , wherein the aggregated ETF passive market data comprises mutual fund data and ETF data.

21 . The method of claim 1 , wherein the second database comprises relational information between the one or more ETFs, indices, and assets.

22 . The method of claim 1 , wherein the time series data comprises one or more of pricing data, market capitalization data, shares outstanding, volume data, derivative exposure data, swaps data, counter party risk data, liquidity data, leverage data, or other market or publicly reported data on the ETF or asset level.

23 . The method of claim 1 , wherein calculating the pass through data comprises calculating an internalization of create redeem activity by broker dealers, operational shorting activity, or other activities down to the authorized participant level for a given ETF.

24 . The method of claim 1 , wherein the analytics layer is configured to produce an analytics output to calculate predicted price slippage caused by passive activity and present the predicted price slippage to the user through the visual display.

25 . The method of claim 1 , wherein the second database is utilized to train a machine learning algorithm or a large language model.

26 . The method of claim 1 , wherein the analytics layer is configured to track one or more mutual fund to ETF conversions.

27 . The method of claim 1 , wherein the analytics layer is further configured to produce an analytics output to generate an actionable recommendation or make a prediction to hedge against risk for a portfolio.

28 . The method of claim 1 , wherein the visual interface is configured to display the generated bidirectional, time-series data structures, and a visual representation of the pass through data calculations mapped onto the bidirectional, time-series data structures.

29 . A computer-implemented system comprising at least one processor, a memory, and instructions executable by the at least one processor to cause the at least one processor to perform operations for providing passive market data transparency using distributed computing architecture, the operations comprising:

a) generating a first database, wherein generating the first database comprises

i) aggregating passive market data from one or more sources, wherein the passive market data comprises index data for one or more indices, exchange traded fund (ETF) data for one or more ETFs, and asset data for one or more assets, where the aggregated passive market data comprises at least time-series data structured as one or more tensors with a plurality of indices, with a tensor represented as a multi-dimensional array data structure,

ii) converting the aggregated passive market data from a non-standard format into a standard format, wherein the standard format is based on a taxonomy of the one or more indices, the one or more ETFs, and the one or more assets across the one or more sources, and

iii) generating a standard identifier according to the taxonomy for each of the one or more indices, the one or more ETFs, and the one or more assets, wherein the standard identifier is linked to all identifiers used for the same asset, index, or ETF;

b) inputting the one or more tensors with the plurality of indices to a machine learning model to predict bidirectional pass through data between at least an asset and an ETF or between at least an index and an ETF in an ETF/index/asset network, with the machine learning model including a multi-dimensional representation of relational structure between the asset and the ETF or the index and the ETF as one or more bidirectional, time-series data structures, wherein the ETF/index/asset network comprises the relationships and associated non-linear interactions of one or more ETFs, one or more indices and one or more assets,

wherein the bidirectional passthrough data comprise one or more characteristics, events or flows, the machine learning model comprising:

a first class of nodes and a second class of nodes,

wherein one or more of the nodes in the first class represent at least first time-series data for a given bidirectional pass through data, with the first time-series data being ETF data,

wherein one or more of the nodes in the second class represent at least second time-series data for the given bidirectional pass through data, with the second time-series data being index data or asset data first directional edges connecting nodes in the first class to nodes in the second class, each first edge specifying a first weight representing a first dependency between a node in the first class and a node in the second class;

second directional edges connecting nodes in the second class to nodes in the first class, each second edge specifying a second weight representing a second dependency between a node in the first class and a node in the second class;

wherein a bidirectional structure of the one or more bidirectional, time-series data structures enables frame of reference translation between nodes in the first class and nodes in the second class;

outputting, from the machine learning model, vectorized computations of bidirectional dataflow between nodes in the first and second classes;

based on the output vectorized computations, training the machine learning model to predict with increased accuracy bidirectional pass through data between nodes in the first and second classes of the machine learning model;

wherein the vectorized computations comprise parallelization of computing the bidirectional dataflow over all indices of at least one of the one or more tensors; and

wherein the distributed computing architecture is configured to perform vectorization that handles dependence and recurrence between indices of the one or more tensors, including permitting a time indexed value associated with a given tensor index to depend on a value associated with a prior time index without explicit iteration over indices and without referring to a specific time-coordinate, by executing one or more sub-vectorized dependency management algorithms that perform concurrent calculation of dependent variable relationships;

c) applying, in the distributed computing architecture, the trained machine learning model over at least the time-series data to generate pass through data based at least in part on values of the bidirectional pass through data;

wherein applying the trained machine learning model comprises executing the one or more sub-vectorized dependency management algorithms to concurrently calculate dependent variable relationships in the time-series data without sequentially iterating over indices of the one or more tensors;

d) storing, in a storage device, the trained machine learning model;

e) generating a second database by aggregating the pass through data, and converting the pass through data to a standard format, wherein the standard format is based on a taxonomy of the one or more generated bidirectional, time-series data structures based at least in part on the bidirectional pass through data and their pass through effects, and generating a standard identifier according to the taxonomy,

wherein the second database is searchable by the standard identifiers;

f) providing an analytics layer for synthesizing the first database, the one or more generated bidirectional, time-series data structures including the bidirectional pass through data, the pass through data, and the second database, and producing an analytics output, wherein producing the analytics output comprises:

i) receiving a search query to search, the one or more generated bidirectional, time-series data structures including the bidirectional pass through data, the pass through data, or the second database, mapping the search query to one or more of the standard identifiers, and returning a set of pass through data across the one or more dimensions of the multi-dimensional ETF/index/asset network, for the one or more standard identifiers of the search query, and

ii) performing one or more tasks using the returned set of passive market data or the set of pass through data, wherein the one or more tasks comprise: mapping passive market activity, performing portfolio risk monitoring, performing direct indexing, performing compliance monitoring, performing conflict monitoring, tracking information flows, or a combination thereof; and

g) generating a passive market visual interface for navigating and interpreting the analytics outputs, wherein the passive market visual interface is configured to be presented on a display, and wherein the passive market visual interface comprises at least one of a market overview window, a forecast and recommendation window, an event and flows window, an ownership window, a portfolio performance, or a risk window.

30 . Non-transitory computer-readable storage media encoded with instructions executable by one or more processors to provide a passive data transparency application comprising:

a) a software module generating a first database, wherein generating the first database comprises

i) aggregating passive market data from one or more sources, wherein the passive market data comprises index data for one or more indices, exchange traded fund (ETF) data for one or more ETFs, and asset data for one or more assets, where the aggregated passive market data comprises at least time-series data structured as one or more tensors with a plurality of indices, with a tensor represented as a multi-dimensional array data structure,

ii) converting the aggregated passive market data from a non-standard format into a standard format, wherein the standard format is based on a taxonomy of the one or more indices, the one or more ETFs, and the one or more assets across the one or more sources, and

iii) generating a standard identifier according to the taxonomy for each of the one or more indices, the one or more ETFs, and the one or more assets, wherein the standard identifier is linked to all identifiers used for the same asset, index, or ETF;

b) a software module inputting the one or more tensors with the plurality of indices to a machine learning model to predict bidirectional pass through data between at least an asset and an ETF or between at least an index and an ETF in an ETF/index/asset network, with the machine learning model including a multi-dimensional representation of relational structure between the asset and the ETF or the index and the ETF as one or more bidirectional, time-series data structures,

wherein the ETF/index/asset network comprises the relationships and associated non-linear interactions of one or more ETFs, one or more indices and one or more assets, wherein the bidirectional passthrough data comprise one or more characteristics, events or flows, the machine learning model comprising:

a first class of nodes and a second class of nodes,

wherein one or more of the nodes in the first class represent at least first time-series data for a given bidirectional pass through data, with the first time-series data being ETF data,

wherein one or more of the nodes in the second class represent at least second time-series data for the given bidirectional pass through data, with the second time-series data being index data or asset data first directional edges connecting nodes in the first class to nodes in the second class, each first edge specifying a first weight representing a first dependency between a node in the first class and a node in the second class;

second directional edges connecting nodes in the second class to nodes in the first class, each second edge specifying a second weight representing a second dependency between a node in the first class and a node in the second class;

wherein a bidirectional structure of the one or more bidirectional, time-series data structures enables frame of reference translation between nodes in the first class and nodes in the second class;

outputting, from the machine learning model, vectorized computations of bidirectional dataflow between nodes in the first and second classes;

based on the output vectorized computations, training the machine learning model to predict with increased accuracy bidirectional pass through data between nodes in the first and second classes of the machine learning model;

wherein the vectorized computations comprise parallelization of computing the bidirectional dataflow over all indices of at least one of the one or more tensors; and

wherein the distributed computing architecture is configured to perform vectorization that handles dependence and recurrence between indices of the one or more tensors, including permitting a time indexed value associated with a given tensor index to depend on a value associated with a prior time index without explicit iteration over indices and without referring to a specific time-coordinate, by executing one or more sub-vectorized dependency management algorithms that perform concurrent calculation of dependent variable relationships;

c) a software module applying, in the distributed computing architecture, the trained machine learning model over at least the time-series data to generate pass through data based at least in part on values of the bidirectional pass through data;

wherein applying the trained machine learning model comprises executing the one or more sub-vectorized dependency management algorithms to concurrently calculate dependent variable relationships in the time-series data without sequentially iterating over indices of the one or more tensors;

d) a software module storing, in a storage device, the trained machine learning model;

e) a software module generating a second database by aggregating the pass through data, and converting the pass through data to a standard format, wherein the standard format is based on a taxonomy of the one or more generated bidirectional, time-series data structures based at least in part on the bidirectional pass through data and their pass through effects, and generating a standard identifier according to the taxonomy, wherein the second database is searchable by the standard identifiers;

f) a software module providing an analytics layer for synthesizing the first database, the one or more generated bidirectional, time-series data structures including the bidirectional pass through data, the pass through data, and the second database, and producing an analytics output, wherein producing the analytics output comprises:

i) receiving a search query to search, the one or more generated bidirectional, time-series data structures including the bidirectional pass through data, the pass through data, or the second database, mapping the search query to one or more of the standard identifiers, and returning a set of pass through data across the one or more dimensions of the multi-dimensional ETF/index/asset network, for the one or more standard identifiers of the search query, and

ii) performing one or more tasks using the returned set of passive market data or the set of pass through data, wherein the one or more tasks comprise: mapping passive market activity, performing portfolio risk monitoring, performing direct indexing, performing compliance monitoring, performing conflict monitoring, tracking information flows, or a combination thereof; and

g) a software module generating a passive market visual interface for navigating and interpreting the analytics outputs, wherein the passive market visual interface is configured to be presented on a display, and wherein the passive market visual interface comprises at least one of a market overview window, a forecast and recommendation window, an event and flows window, an ownership window, a portfolio performance, or a risk window.