Data integration is increasingly critical to companies’ ability to win, serve and retain their customers. Enterprises face increasing data integration challenges with the growing volume of data across hybrid and multiple clouds.
To build up an effective data and analytics architecture, enterprises should look at leveraging new approaches in data integration. In this article, we take a look at four emerging data technologies to consider for future data integration projects.
Content Summary
Data as a service (DaaS)
Data Mesh
Knowledge Graph
Query Accelerator
Data as a service (DaaS)
Data as a service (DaaS) – also known as data as a product (DaaP) – delivers a common data access layer through application programming interfaces (APIs), SQL, ODBC/JDBC and other protocols, leveraging data platforms such as data virtualisation, data mesh, integration platform as a service (iPaaS) and others. It focuses on a common data access layer to support querying, reporting, data access, and integrated and custom-built applications, offering several business benefits, including supporting a common view of business and customer data using industry-standard protocols. It is likely to continue growing, with further innovation in real-time updates, integration and self-service capabilities.
Data Mesh
A data mesh offers the ability to optimise mixed workloads by matching processing engines and data flows with the right use cases. It interfaces to the event-driven architecture, enabling support for edge use cases. It matches the data, queries and models to the solution to keep each party – human and machine – in sync and speaking the same language. Besides still being in its infancy, we are likely to see data mesh evolve into a platform in the long term.
Knowledge Graph
A knowledge graph makes use of graph engines to support complex data connections and integration. It helps build recommendation engines, cleanse data, perform predictive analytics and connect data quickly. Developers, data engineers and data architects can rapidly work through messy, unrelated data to accelerate app development and new business insights. As a data integration technology, knowledge graphs are still evolving with support for automation, built-in AI/machine learning and self-service capabilities.
Query Accelerator
Query accelerator helps developers and data engineers optimise queries quickly and move compute closer to data, thus minimising data movement. This technology is helpful when you have data stored in data lakes, object stores or complex data warehouses where tuning queries are not often straightforward. Unlike data virtualisation systems, query accelerators speed up queries through an improved query optimiser, fetching only selected data from data sources. It is expected to evolve further in the coming years, with improved AI/machine learning and data intelligence being built in, combined with automated integration of distributed data.
