Methods and systems for optimizing scattered databases and accessing complex data structures for end-user integrated platforms
The present invention relates to the field of digitally accessing, modeling and optimizing scattered ‘big-data’ from various assets having complex and dynamic physical relationships. The present disclosure also relates to a system and method for supporting version-of-the-truth from multiple data sources and for multiple disciplines. It helps organizations that possess various physical assets that are been utilized by multiple user groups and through a period of time to optimize operations, conduct surveillance and manage complex business processes related to their valuable assets and equipment. The invention is built with an open architecture to connect to various corporate databases combining real-time and relational data. The system employs exception-based surveillance, sophisticated queries, and no-code methods to automatically detect deviations in asset performance from optimal conditions, and flag them to the right users, at the right time, and in the right way. The present invention also allows monitoring the status and performance of equipment and facilities by presenting its findings through intuitive dashboards and automated workflows. In addition, a mobile version of the present invention is developed to help users execute day to day work as well as manage and maintain equipment and assets more efficiently.
1. A computer implemented method for data collection, integration and presentation, comprising:
providing a data warehouse, a data abstraction layer, a data integration layer, and a graphical user interface, wherein a set of information is exchanged between said data warehouse and said data abstraction layer, and between said data abstraction layer and said data integration layer;
storing time series and relational data points collected from physical assets in said data third-party data warehouses or source databases;
configuring collected data points into digital twins each representing a physical asset;
running queries to request a single bulk of data through a data abstraction layer that is integrated to a data consumer;
combining queries into a query bulk to reduce the request frequency of the application programming interface;
utilizing said query bulk to fetch and stream asynchronous records from said abstraction layer to said data consumer;
enabling the presentation of multiple asynchronous events and streaming of said events to the data consumer; and
displaying a graphical representation of said multiple asynchronous events using said graphical user interface,
wherein said data consumer performs the following steps:
providing a set of read and write queries,
generating a read query to collect data from an organizational data warehouse to provide analysis and reports;
generating a write query to change the records obtained by said read query wherein such query is configured to change the records in the source application database;
showing updated records through a new read query; and
re-routing said read query automatically to trigger the new read query to the source application database instead of the data warehouse, such that corrected and updated records are returned to the data consumer.
2. The method of claim 1 , further comprising:
recording changes to time series-data or relational records that represent a plurality of physical assets in a database;
accessing said database to generate new tags to physical assets replacements;
verifying whether said new tags have the same function and role as those stored in the data warehouse;
configuring said new tags through a new attribute version identifier when new tags have the same function and role as those stored in the data warehouse, and binding all data from different attribute versions identifiers to report the complete dataset; and
configuring said new tags through a new attribute identifier when said new tags do not have the same function and role as those stored in the data warehouse, and reporting changes to time-series data points for a single attribute identifier.
3. The method of claim 1 , wherein said digital twins comprising one functional element and several variable child elements that are configured within the digital twin templates, wherein one template is created for all elements which have the same function with optional attributes where applicable, analysis attribute configuration for motor performance is configured at least one time without considering other optional attributes, and different results based on the optional attributes are returned by using analysis formulas.
4. The method of claim 1 , further comprising:
creating a hierarchy of source tag attributes; and
configuring said digital twin elements to have multiple source tag attributes; wherein data consumer hierarchies comprising of a dynamic sorting criteria for all sorts of data originating from data warehouse and external structured databases, and
said data integration layer keeps unified digital twin attributes in the database ensuring that the same version of information is exchanged with search and read queries.