Azure Data Factory Posts Collection
Azure Data Factory (ADF) is a powerful ETL tool on Azure cloud. here is a collection of posts I wrote about ADF on Madeira blog:
Working with templates in Azure data factory
Using pipeline templates in ADF makes is easy to reuse code and move it to other ADF deployments.
https://www.madeiradata.com/post/working-with-templates-in-azure-data-factory
The wind of change – change data capture in data factory
Change data capture (CDC) is used to synchronize changes between systems. See how ADF uses CDC to read from source systems or files.
https://www.madeiradata.com/post/the-wind-of-change-change-data-capture-in-data-factory
Data Factory – Move files to time folder structure
Organizing files into time structure, where the top level folders are for years, than months, than days, is very useful in datalakes. see how to create that structure with ADF.
https://www.madeiradata.com/post/data-factory-move-files-to-time-folder-structure
Azure data factory – clear out the garbage
How to clear garbage created by inconsistencies between ADF environments and\or Git connection.
https://www.madeiradata.com/post/azure-data-factory-clear-out-the-garbage
Comparing Azure ETL tools: Azure data factory vs Azure Databricks
Comparing the ETL part of ADF vs. Databricks
https://www.madeiradata.com/post/comparing-azure-etl-tools-azure-data-factory-vs-azure-databricks
Disable activities on Azure Data Factory
A new feature that allows you to disable parts of your pipeline.
https://www.madeiradata.com/post/disable-activities-on-azure-data-factory
Azure data factory self-hosted integration runtime. what is it and when to use it?
ADF self-hosted integration runtime is used to allow ADF to connect to data sources inside your company firewall, but it also has other uses.
https://www.madeiradata.com/post/azure-data-factory-self-hosted-integration-runtime-what-is-it-and-when-to-use-it
Change data capture on Azure data factory revisited
About change data capture on ADF part 2.
https://www.madeiradata.com/post/change-data-capture-on-azure-data-factory-revisited
Data Factory error handling with finally
How do you use error handling in ADF, and what do you do when you need an activity to run on either failure or success?
https://www.madeiradata.com/post/data-factory-error-handling-with-finally
How to avoid variable mix-ups in Data Factory for-each loop?
Using variables in ADF for-each loop might cause mix-ups. See how to avoid it.
https://www.madeiradata.com/post/how-to-avoid-variable-mix-ups-in-data-factory-foreach-loop