Your browser is out of date

Update your browser to view this website correctly. Update my browser now

×

Overview
 

Address operational challenges around deploying and monitoring of NiFi flows.

Cloudera DataFlow for the Public Cloud is a new experience on Cloudera Data Platform (CDP) that addresses the operational challenges around deployment, scaling, and infrastructure sizing for Apache NiFi flows and offers comprehensive monitoring capabilities.

DataFlow for Public Cloud addresses the following challenges:

  • Sharing resources amongst multiple NiFi flows in the same cluster impacts the overall performance

  • Underestimating cluster sizes can result in unplanned infrastructure expense

  • Scaling up the infrastructure on an as-needed basis can be an operational nightmare

  • Monitoring data flow metrics across multiple clusters in a single view is not possible with current tooling

Read how DataFlow for the Public Cloud eliminates these issues

 

 

CDF for Public Cloud diagram

Capabilities

  • Pre-built NiFi ReadyFlows
  • Centralized Dashboard
  • Flow Catalog

Pre-built NiFi ReadyFlows


Boost productivity with common NiFi flows pre-built for new and expert NiFi users.

ReadyFlows are flow definitions to help new NiFi users to get started with the most common data flow use cases and successfully implement them with Cloudera DataFlow. Users can download the flow definition, quickly configure properties, import into the Catalog, and then deploy into their public cloud environment.

Examples of ReadyFlows:

  • Consume Kafka to Kafka

  • Consume from Kafka and filter data

  • Consume Kafka to S3 bucket

DataFlow ReadyFlow Screenshot

Centralized Dashboard


Monitor all your NiFi flows across multiple cloud clusters from a single dashboard.

The Dashboard page within the Cloudera DataFlow user interface is the central monitoring console for all NiFi flow deployments across environments. Users can monitor flow metrics and infrastructure usage as well as check status of deployments ensuring they meet performance expectations. Users gain insights into the health of deployments by monitoring the status of data sent and received as well as checking on any alerts on KPIs they have defined.

DataFlow Monitoring screenshot

Flow Catalog


Store, manage, and deploy all your flow definitions from a single place.

The Catalog, another important page in the Cloudera DataFlow user interface, is where the flow definitions are stored. The user can manage the CDF flow definition lifecycle from import to versioning and deletion. Importing a flow into the catalog allows the user to get a traditional NiFi flow definition set up for configuration and deployment in the public cloud. Versioning capabilities of the Catalog also enable the user to deploy specific versions of a flow. 

Catalog Screenshot

Key features

Scale up and down flow deployments on Kubernetes clusters managed by CDP,  based on CPU utilization within the boundaries specified in the deployment wizard. 

Predefined set of data flows called ReadyFlows help developers implement the most common data flow use cases. Boosts developer productivity to deploy pre-built flows quickly with minimal configuration.

Flow definitions are stored in the catalog. Manage the CDF flow definition lifecycle from import to versioning and deletion. The catalog is also where users can initiate new flow deployments.

A simple step-by-step process turns a flow definition in the catalog into a flow deployment for processing data. Enable users to configure parameters, create guardrails for auto-scaling, and set KPI definitions for the flow deployments.

Monitor different flow deployments across multiple clusters at a glance in real time. Deployment details in the dashboard show pre-defined KPIs, system metrics, as well as system events and alerts.

Support for Kubernetes in the public cloud environment enables isolation of individual flow deployments and allows scaling of each cluster without resource contention. This also helps with troubleshooting and isolating failures.

Ready to take a deeper look?


Experience DataFlow for the Public Cloud for yourself

Get started

PRODUCT DOCUMENTATION

Read about technical specifications, architecture, tutorials, and how to articles about Cloudera DataFlow for the Public Cloud.

Learn more

CDF FOR THE PUBLIC CLOUD PRICING


Evaluate Cloudera DataFlow for the Public Cloud pricing across various cloud instance types

Get the details

DATAFLOW OVERVIEW TOUR

Get a hands on tour of Cloudera DataFlow for the Public Cloud.

Access now

CLOUDERA COMMUNITY WITH NIFI

Connect with your peers, ask questions, troubleshoot, and learn more about Apache NiFi

Explore now

NIFI TRAINING

Book a three day hands-on training course on Apache NiFi fundamentals and more.

Go learn

PRODUCT WEBINAR

Watch the introduction and demonstration of Cloudera DataFlow for the Public Cloud.

Go watch

Datasheet

Cloudera DataFlow for the Public Cloud

Ebook

Apache NiFi for dummies

Solution Brief

Data-in-motion philosophy: A blueprint for enterprise-side streaming data architecture

World-class training, support, & services

Your form submission has failed.

This may have been caused by one of the following:

  • Your request timed out
  • A plugin/browser extension blocked the submission. If you have an ad blocking plugin please disable it and close this message to reload the page.