Cloudera named a leader in 2022 Gartner® Magic Quadrant™ for Cloud Database Management Systems Get the report

Apache Slider

A Framework for YARN-based, Long-running Applications In Hadoop

Apache™ Hadoop continues to attract new engines to run within the data platform, as organizations want to efficiently store their data in a single repository and interact with it simultaneously in different ways. They want SQL, streaming, machine learning, along with traditional batch processing…all in the same cluster. Many of these applications must be “always-on” or “long-running” services that are ready to process data whenever it comes in.

Slider “slides” these long-running services (like Apache HBase, Apache Accumulo and Apache Storm) onto YARN, so that they have enough resources to handle changing amounts of data, without tying up more processing resources than they need.

What Slider Does

Slider is a framework for deployment and management of these long-running data access applications in Hadoop.

Slider leverages YARN’s resource management capabilities to deploy those applications, to manage their lifecycles and scale them up or down–even while the application is running. Slider “slides” existing long-running services (like Apache HBase, Apache Accumulo and Apache Storm) onto YARN, so that they have enough resources to handle changing amounts of data, without tying up more processing resources than they need.

Apache Slider allows users to create and run different versions of heterogeneous long-running applications in Hadoop with YARN. Each application instance can be configured differently, with its operational life cycle managed individually. On an on-demand basis, Slider can expand or shrink application instances while they are running. In the case of container failure, Slider transparently leverages YARN facilities to manage application recovery. All of this is available on Linux or Windows platforms.

These Apache Slider features provide three key benefits to enterprises running Hadoop:

Benefit Description
Turnkey YARN enablement Enables long-running applications to take advantage of YARN’s benefits without code changes:
  • Availability - YARN works with the application in Slider to recover or restart
  • Flexibility - YARN lets the application in Slider scale up or scale down
  • Resourcing - YARN allocates cluster resources and scheduling of work for long-running applications across a Hadoop cluster
Hadoop integration Applications running with Apache Slider cooperate with the Enterprise Hadoop ecosystem in an integrated way--leveraging Hadoop’s data and processing resources, as well as its security, governance, and operations capabilities
Lifecycle management Automatically makes applications manageable through Apache Ambari without any additional work

How Slider Works

Apache Slider views any application, as a set of components and each component is a daemon or executable with its own configuration, scripts, and data files. Components may have one or more instances. Slider manages applications by managing their component instances.

To manage application component instances, Slider launches a YARN application master for each instance. After the launching an application master, Slider can allocate or de-allocate resources and stop or start an application instance. This can be done based on the application admin’s request through the Slider client or through YARN’s resource scheduling pre-emptions.

Your form submission has failed.

This may have been caused by one of the following:

  • Your request timed out
  • A plugin/browser extension blocked the submission. If you have an ad blocking plugin please disable it and close this message to reload the page.