Organizations all over the world are migrating their IT infrastructures and applications to the cloud. Organizations are leveraging cloud analytics to extract useful insights from big data, which draws from a variety of sources such as mobile phones, Internet of Things (IoT) sensors, and various software-as-a-service (SAAS) products such as customer relationship managers (CRMs). Enterprises and businesses need to scale and modernize their data architectures to unify structured and unstructured data in a logical fashion, to cope with the constant demand to stay competitive.

Inundated with Data

  • The vast amount of data is still in siloes
  • The requirement to democratize data, by enabling consistent and trustworthy data flow management in the cloud
  • Data security and governance concerns, when it comes to integrating the data

Organizations need a solution that facilitates tighter data integration in the form of DataOps (people, tools, and culture), such as the ability to facilitate the implementation of a modern data warehouse in the cloud, with minimal data movement, and secured access. This is possible via a common virtual data access/service layer, enabling real-time analytics with minimal data movement and zero data replication.

Microsoft Azure Synapse, and the Denodo Platform, working together, can enable this. These technologies can accelerate the quality of insights that can be generated, without sacrificing data security and privacy, advancing the user towards advanced machine learning (ML) and artificial intelligence (AI).

Leveraging Azure Synapse and the Denodo Platform to Enhance Business Agility and Insights

In parallel, the data virtualization capabilities of the Denodo Platform extends the value of Azure Synapse by quickly enabling a logical data warehouse, combining a variety of data sources into a virtual data layer that can be easily leveraged by consuming applications such as BI tools (e.g. Power BI), web services, and lot more. This is accomplished without moving or replicating any data. The Denodo Platform provides a single point for access management and security, as well as a built-in data catalog that enables users to tag, search for, and easily find business objects that are worthy of advanced analytics.

A typical Azure Synapse workflow consists of data that flows in from various sources, which is stored in Azure data lake storage -gen 2 (ADLS2). Some data might also be stored in Cosmos DB, Blob storage, and other SaaS based applications that are outside of the Synapse analytics platform.

The Denodo Platform can be positioned as the central semantic layer that captures all the metadata across Azure Synapse and surrounding data sources to present a solid architecture framework that supports a variety of use cases, as described below.

The Benefits of the Denodo Platform with Azure Synapse

A recommended best practice is to always deploy a Denodo Platform instance as close as possible to the data sources. Smart, built-in query optimization determines the optimal approach for speed and cost, enabling the processing to be pushed down to the database, to minimize performance overhead and network latency.

The Denodo Azure Synapse connector offers additional benefits:

  • Seamless Interoperability: Native support for Azure Synapse provides optimal query generation and enables customers to take advantage of advanced features like windowing and statistical functions, to guarantee optimal query push-down.
  • Advanced Optimization: Support for techniques like data movement (data shipping) that enable multi-pass executions based on intermediate, temporary tables.
  • High-Performance Caching: Use of Azure Synapse as a high-performance caching database for the Denodo Platform in the cloud.
  • Frictionless Data Movement: The ability to leverage Azure Synapse APIs with the Denodo Platform to enable customers with frictionless data movement to Azure Data Lake Storage (ADLS gen2).
  • Metadata Integration: Integration with Azure Synapse metadata catalog provides the metrics to outline an optimally efficient query plan.

Getting Started

Denodo Platform offerings are also available on the Azure Marketplace. You can choose between offerings based on the number of data sources you need to integrate in the cloud.

Originally published at https://www.datavirtualizationblog.com on September 16, 2020.

We do #DataVirtualization We care about #AgileBI, #BigData #Analytics, #Dataservices, #DataManagement, Logical #DataWarehouse Web, #SaaS and #Cloud integration.