CDP Data Developer Exam Guide CDP-3001

Connect and move data between systems

• Use Apache Airflow to connect and download using Rest APIs
• Use NIFI to ingest data from various systems
• Use NIFI to send and fetch data between HDFS and S3
• Use Apache Spark and Nifi to load data into Apache Kudu

Build and manage a data warehouse

• Use Impala to define tables and views
• Use HDFS to store data
• Use HIVE DDL to work with partitioned tables
• Persist data mart from views into materialized views

Build, schedule, execute, and monitor data pipelines

• Use Apache Airflow to schedule ETL pipelines
• Use Apache Spark for Extract, load and transform data
• Use Apache Nifi to schedule data pipelines and transform data with processors.

Clean and serve data to the end-users

• Use Spark to read, convert and combine files
• Create joins and cleanse data with Spark
• Work with badly formatted dates with Spark
• Use Impala to provide custom data marts
• Work with Kudu and Parquet storage formats

Perform data quality checks

• Use Spark to detect anomalies in data
• Use Impala for performing data quality checks
• Check file conditions with Hadoop shell commands
• Use Hive DDL to create partitioned tables
• Use Apache Airflow to schedule quality checks

Debug data issues reported by end-users

• Use Impala and Spark to find data discrepancies
• Cross check raw data and processed data

Data backup and disaster recovery

• Use NiFi registry to create backups
• Backup using Cloudera Data Engineering