Creating a Denodo Data Virtualization Layer for Illumina’s Global Quality Team

Illumina, a San Diego-based biotech company, tasked the company that I work for, Analytica Consulting, with creating a Denodo data virtualization layer to support the data access needs of Illumina’s Global Quality Operations department. Now that the implementation is complete, engineers, analysts, and users from the Global Quality team have access to real-time quality data including up-to-date information on complaints, non-conformances, corrective actions, supplier quality reports, audits, and more.

Now, enterprise data from a variety of disparate systems can be integrated and delivered to business users on demand. Because of this, users are able to unlock new insights from combining and exploring relationships between data sets from a quality management system and many other related domains including manufacturing, safety, facilities, and more.

The Journey to a Seamless Data-Access Layer

One of the challenges along the way was creating base view connections to an OData 3.0 cloud data source using preemptive authorization. We worked closely with the Denodo support team to find a solution using Denodo’s OData 2.0 custom wrapper.

Analytica implemented the project in three broad phases:

1. Replicating and testing hundreds of views from legacy systems into Denodo virtual databases
2. Making architectural and optimization recommendations based on industry best practices, exceeding Illumina’s performance expectations
3. Creating a repository of system views, intermediate views, and business views that can be leveraged by developers to answer future Illumina quality questions not yet asked

Significant Results

This project had a significant impact on engineers, analysts, and other users who now have access to real-time quality data, including up-to-date non-conformances, corrective actions, complaints, audits, and more. Prior to this, analysts had limited ability to combine data across disparate data sources. In addition to being able to use OData 2.0 custom wrappers to connect to OData APIs, other connections were made to JDBC sources such as Microsoft SQL Server. With the development of the data virtualization layer, analysts and developers now have the foundation and means to create new dashboards and reports using validated data without having to worry about the complexities of the underlying data source connections.

Leveraging Data Virtualization

If your company is looking for a way improve data access across the board, to improve the quality of insights across the enterprise, you might take a serious look at data virtualization.

Originally published at http://www.datavirtualizationblog.com on December 5, 2019.