Databricks Lakeflow Connect is a fully managed data ingestion solution from Databricks, providing a simple, efficient connector to ingest data from a wide range of sources. Use this new functionality to stream data from SQL Server into Databricks Data Intelligence Platform for data processing and analytics utilizing powerful Apache Spark Clusters.
Lakeflow SQL Server Connecter supports a wide range of SQL Server database variations, including Microsoft Azure SQL Database, Amazon RDS for SQL Server, Microsoft SQL Server running on Azure VMs and Amazon EC2, and on-premises SQL Server accessed through Azure ExpressRoute or AWS Direct Connect.
Lakeflow can be integrated with SQL Server using Microsoft change tracking (CT) or Microsoft Change Data Capture (CDC) enabled to support efficient, incremental ingestion. CDC provides historical change information about insert, update, and delete operations, and when the actual data has changed. Change tracking identifies which rows were modified in a table without capturing the actual data changes themselves. The connector captures an initial load of historical data on the first run of your ingestion pipeline. Then, the connector tracks and ingests only the changes made to the data since the last run, leveraging SQL Server's CT/CDC features to streamline operations and efficiency.
|
|
|
|
|
|