Your browser is out of date

Update your browser to view this website correctly. Update my browser now

×

Overview

What is Edge and Flow Management?

Cloudera DataFlow (CDF) is a real-time streaming data platform for managing your data from edge to cloud. One of the key tenets of CDF is Edge and Flow Management.

The Flow Management capabilities of CDF, powered by Apache NiFi, deliver high-scale data ingestion, transformation, and management to enterprises from any-to-any environments. These address key enterprise and hybrid cloud use cases like data movement; log data ingestion; and acquisition of all types of streaming data including social, mobile, clickstream, and IoT data.

The Edge Management capabilities of CDF are made up of edge agents (MiNiFi) and an edge management hub called Edge Flow Manager. It manages, controls, and monitors edge agents to collect data from edge devices and push intelligence back to the edge. This addresses IoT use cases such as predictive maintenance, fleet management, and asset tracking.

Use cases

  • Predictive maintenance
  • Patient monitoring
  • Data movement

Predictive maintenance


Lower costs and reduce downtime with predictive maintenance. 

Predictive Maintenance is a data-driven approach to analyze IoT and sensor data from connected equipment to effectively predict when and how an asset might fail, detect variances, understand warning signals, and quickly identify patterns that might indicate a potential breakdown. Cloudera DataFlow’s Edge and Flow Management capabilities modernize and simplify data ingestion from hundreds of connected assets to enhance predictive maintenance.

Patient monitoring


Capture real-time feeds from patient-monitoring devices to detect anomalies.

Biometric and telemetric devices are used in healthcare organizations to monitor post-surgery or high-risk patients. Ingesting sensor data from these devices about various patient vitals helps detect abnormalities or concerning patterns. Cloudera DataFlow’s Edge and Flow Management helps capture patient-monitoring data and delivers them to stream-processing engines for insights.

Data movement


Connect, integrate, and move massive volumes of data across hybrid and multi-cloud environments.

Traditional ETL processes are for use cases where data must move from one database to another. Modern enterprises transfer data from on-premises to cloud or cloud-to-cloud, moving petabytes of information in a matter of just hours. Cloudera DataFlow’s Flow Management capabilities are purpose-built for such use cases.

Rabobank: Facilitating financial independence through real-time data insights

8.3 million customers worldwide.

Read the success story

Key features

Hundreds of prebuilt processors are available to connect with a range of data sources, devices, and protocols. The user interface allows you to build sophisticated data flow pipelines with drag-and-drop ease.

Understand the origin and attribution of data as it moves throughout the enterprise, empowering the governance team to explain how any data point is affected by any system. Data lineage information is generated for everything it does at a fine-grained level, even when records change before and after an event. 

Ingest, capture, and deliver data in real-time from any streaming source, including clickstreams, social media, mobile, or IoT devices. Enable actionable insights by easily connecting, transforming, managing, and monitoring them using complex data flow applications with NiFI’s 450+ processors and a custom-build monitoring dashboard. 

Handle any throughput by moving petabytes of data from one data center to another in just a few hours or move data from your on-premises environment to the cloud or vice versa. Enable a multi-cloud model with a cloud-vendor-agnostic approach to managing data. 


Adopt a DevOps-style data flow development lifecycle with NiFi Registry to deliver your flow applications faster and deploy them easily from one environment to another. Enable your development team to version their data flows and set up promotion schemes across environments.

Enable edge management at scale with command, control, and monitoring of hundreds of thousands of agents with minimal footprint to collect, filter, and process data. Allow end-to-end machine learning algorithms at the edge with automated learning loops.

Deployment options

Ingest and deliver data from the edge to the cloud across edge, on-premises, public cloud, and hybrid cloud environments.

Flow Management for Data Hub

  • Spin up NiFi and NiFi Registry into your public cloud with Flow Management for Data Hub 
  • Choose your cluster size (S/M/L) and launch it across AWS, Azure or GCP
  • Extend the same on-premises user experience into the public cloud
     

DataFlow for the Public Cloud

 
  • Monitor all NiFi flows across multiple cloud clusters from a single dashboard
  • Boost productivity by leveraging pre-built NiFi flows from a gallery of ReadyFlows
  • Optimize the infrastructure sizing by allowing the DataFlow experience to auto-scale the NiFi flows
     

Get started

Product documentation

Read about technical specifications, architecture, tutorials, and how-to articles about Apache NiFi.

Learn more

CDP Data Hub pricing


Evaluate CDP Public Cloud pricing for Data Hub across various instance types and cloud providers.

Get the details

Logging Modernization

Learn more about one of the common use cases of flow management— Logging Modernization.

Read now

NiFi on the cloud

Extend your flow management capabilities to the cloud with CDP Data Hub. 

Watch a quick intro video

Cloudera Community on NiFi

Connect with your peers, ask questions, troubleshoot, and learn more about Apache NiFi.

Explore now

Training

Book a three day hands-on training course on Apache NiFi fundamentals and more.

Go learn

Analyst Report

451 Research: Cloudera updates DataFlow for IoT data processing at the edge

Ebook

Apache NiFi for Dummies

Webinar

Edge management: Gaining value from IoT

Solution Brief

Data-in-motion philosophy: A blueprint for enterprise-wide streaming data architecture

World-class training, support, & services

Your form submission has failed.

This may have been caused by one of the following:

  • Your request timed out
  • A plugin/browser extension blocked the submission. If you have an ad blocking plugin please disable it and close this message to reload the page.