In this demo video, Philip Zeyliger, a software engineer at Cloudera, discusses the Activity Monitoring and Operational Reports in Cloudera Manager.

Activity Monitoring

The Activity Monitoring feature in Cloudera Manager consolidates all Hadoop cluster activities into a single, real-time view. This capability lets you see who is running what activities on the Hadoop cluster, both at the current time and through historical activity views. Activities are either individual MapReduce jobs or those that are part of larger workflows (via Oozie, Hive or Pig).

Activity Monitoring provides many statistics – both in tabular displays and charts – about the resources used by individual Hadoop jobs and at the aggregate cluster level. The Comparison feature in Activity Monitoring shows the performance of the selected Hadoop job compared with the performance of other similar Hadoop jobs.

In this demo video, BC Wong, a software engineer at Cloudera, discusses the Hadoop Service Monitoring feature in Cloudera Manager. Service Monitoring helps you monitor and manage your Hadoop clusters effectively.

Through the Service Monitoring feature, customers can monitor dozens of service health and performance metrics about the overall service (HDFS, MapReduce, HBase). They can also examine underlying role instances (Namenode, Datanodes, JobTracker, TaskTrackers, Region Servers etc.) in your Hadoop cluster and see what’s going wrong – or what is about to go wrong.

Service Monitoring presents health and performance data in a variety of formats including interactive charts through Cloudera’s new, enhanced user interface. Every Service Monitoring page also includes a widget to enable quick search for relevant Events and Logs associated with the service under consideration. Important Event and Log messages are also highlighted in the various charts. You can also monitor metrics against customizable thresholds, which results in Alerts that operators can pay attention to.

In this demo, Henry Robinson, a software engineer at Cloudera, discusses the Log Management, Event Management and Alerting features in Cloudera Manager that help make sense out of all the discrete events that take place across the Hadoop cluster. He demonstrates how to search the logs valuable information, note important events that pertain to system health and create alerts to warn you when things go wrong.

Log Management

Every process in a Hadoop cluster regularly writes to a log file, which captures valuable data but also creates volumes of information that is difficult to manually sort. Cloudera Manager’s comprehensive log management feature contextualizes all system logs from across the Hadoop cluster and allows the operator to search and filter by service, role, host, keyword and severity. The application also proactively scans the log files for irregularities and warns you before the Hadoop cluster is impacted.

Event Management

With event management, Cloudera Manager proactively reports on important events in the Hadoop cluster such as a change in service health or metrics, log messages with a certain severity or keyword, or abnormal job performance. It creates and aggregates these relevant Hadoop events, and makes them available for searching and alerting.

Alerting

Service and Configuration Management (Part I & II)

We’ve recently recorded a series of demo videos intended to highlight the extensive set of features and functions included with Cloudera Manager, the industry’s first end-to-end management application for Apache Hadoop. These demo videos showcase the newly enhanced Cloudera Manager interface and reveal how to use this powerful application to simplify the administration of Hadoop clusters, optimize performance and enhance the quality of service.

In the first two videos of this series, Philip Langdale, a software engineer at Cloudera, walks through Cloudera Manager’s Service and Configuration Management module. He demonstrates how simple it is to set up and configure the full range of Hadoop services in CDH (including HDFS, MR and HBase); enable security; perform configuration rollbacks; and add, delete and decommission nodes.

Part I

Part I of the Service and Configuration Management demo focuses on managing services and configuring a cluster for optimal performance.  It also demonstrates how to administer users within Cloudera Manager, configure role-based permissions, and better manage security.

The Development track at Hadoop World is a technical deep dive dedicated to discussion about Apache Hadoop and application development for Apache Hadoop. You will hear committers, contributors and expert users from various Hadoop projects discuss the finer points of building applications with Hadoop and the related ecosystem. The sessions will touch on foundational topics such as HDFS, HBase, Pig, Hive, Flume and other related technologies. In addition, speakers will address key development areas including tools, performance, bringing the stack together and testing the stack. Sessions in this track are for developers of all levels who want to learn more about upcoming features and enhancements, new tools, advanced techniques and best practices.

Preview of Development Track Sessions

Building Web Analytics Processing on Hadoop at CBS Interactive
Michael Sun, CBS Interactive

Philip Zeyliger is a software engineer at Cloudera and started the SCM
project.

Two weeks ago, at Hadoop Summit, we released our Service and Configuration Manager (SCM) Express. It’s a dramatically simpler and faster way to get started with Cloudera’s Distribution including Apache Hadoop (CDH). In a previous blog post, we talked in some detail about SCM Express and what it can do for you.

The screencast included in this post demonstrates the simplicity of a CDH installation using SCM Express. The “Directors” conversing in the background are engineers Philip Langdale and Philip Zeyliger and VP of Products, Charles Zedlewski.

Video