Azure Data Factory (ADF): Nested ForEach Activity

In some cases i need to loop folders and files in Azure Data Lake Storage in order to import data in the destination. The file structure can look like: That means that i need to loop this stricture with two nested ForEach Activities in ADF pipeline – one for the […]

Azure: ETL with Synapse Serverless Database

Azure Synapse Analytics offers the so called “serverless” database. It keeps the data in .parquet files. One thing i was suspicious with was the data types in the .parquet file. I decided to test if they are all NVARCHAR(4000) or we can use the standard SQL Server data types. In […]

T-SQL: Hash Different Data Types

Just a quick note: pay attention when you hash different data types. The result is different. Also the lower/upper case: Keep it simple :-)

T-SQL: Avoid MERGE. Write separate statements instead

The MERGE statement was introduced in SQL Server 2008 and the developers embraced it immediately. Later a feedback came and the statement was criticized as not very efficient. Aaron Bertrand collected some links about this here. MERGE statement anatomy The MERGE statement joins ‘source’ and ‘target’ tables and runs INSERT, […]

T-SQL: Avoid Cursor. Use WHILE instead

Set-based Database Development As database developer, we need to switch our mindsets to “Set-based thinking”. In simple words we need to manipulate the data as a “set”. An example of set-based database development is: JOIN tables Aggregate the result Calculate the portion (%) of a row to the total of […]

Azure: Pass secrets to Azure Function via Key Vault

In a few of my previous posts (links at the end of this post), i created Kafka consumer with Python. The same can be done with Azure Function. in this example i show how to pass secrets to the Azure Function with Azure Key Vault. Create Key Vault ‘kv-afkv-test’ in […]

Python: Jupiter Notebooks Development on localhost 1

Download JupyterLab portable from Install and start the app: Open in browser: Add PyPI: Add the code that i posted here and start the development: Related posts: Python: Build JSON Array and keep the last object based on key Keep it simple :-)