In this session on Data engineering and analytics pipeline automation you’ll learn how-to:
1. Filter and prepare data sets to match the requirements discovered in the previous session. 2. Optimize storage and partitioning for the data sets
3. Automate Data Engineering jobs with CDSW; See example jobs running, Learn how to create complex depended jobs and a quick look at the jobs API