Cleaning Data with Spark
Cleaning data is a very common task for data professionals. In this post, I demonstrate a few common data cleaning task with spark Python and SQL.
Category: SQL
SQL Windows Functions might be non-deterministic
How using SQL Windows functions with non unique order column can cause indeterminate results