Building an HVAC System Analysis Application
Overview
NOTICE
As of January 31, 2021, this tutorial references legacy products that no longer represent Cloudera’s current product offerings.
Please visit recommended tutorials:
- How to Create a CDP Private Cloud Base Development Cluster
- All Cloudera Data Platform (CDP) related tutorials
Introduction
Hortonworks Connected Data Platform can be used to acquire, clean and visualize data from heating, ventilation, and air conditioning (HVAC) machine systems to maintain optimal office building temperatures and minimize expenses.
Big Data Technologies used to develop the Application:
- Historical HVAC Sensor Data
- CDF Sandbox
- HDP Sandbox
Goals and Objectives
- Learn to write a shell script to automate development environment setup
- Learn to build a NiFi flow to acquire HVAC machine sensor data
- Learn to write Hive scripts to clean the HVAC machine sensor data and prepare it for visualization
- Learn to visualize HVAC machine sensor data in Zeppelin
Prerequisites
- Downloaded and Installed the latest [Hortonworks HDP Sandbox](https://www.cloudera.com/downloads/hortonworks-sandbox/hdp.html?utm_source=mktg-tutorial Sandbox
- Read through Learning the Ropes of the HDP Sandbox to setup hostname mapping to IP address
- Must have at least 16GB of RAM for HDP Sandbox and 4 GB of RAM for your machine
- Enabled Connected Data Architecture:
Outline
The tutorial series consists of the following tutorial modules:
1. Application Development Concepts: Focus on HVAC fundamentals, common sensors used in HVAC Systems and ways to analyze data from these sensors to understand the status of the HVAC Systems.
2. Setting up the Development Environment: Any Configurations and/or software services that may need to be installed prior to building the data pipeline and visualization notebook.
3. Acquiring HVAC Sensor Data: Create a part of the data pipeline using Apache NiFi to ingest, process and store 1 month of Historical HVAC Sensor Data into HDFS.
4. Cleaning Raw HVAC Data: Create the next portion of the data pipeline using Apache Hive to upload the data into Hive tables, so the data can be cleaned and queried to capture valuable insight about the status of HVAC Systems regulating temperature in buildings across various countries.
5. Visualizing Sensor Data Related To HVAC Machine Systems: Perform data analysis on HVAC sensor data to find the HVAC Building Temperature Characteristics Per Country - keep count of HOT, COLD, NORMAL ranges per country, Extreme Temperature in Buildings Having HVAC Products - building that are experiencing extreme temperature whether it be HOT or COLD even though they have HVAC products.