X

Cloudera Tutorials

Optimize your time with detailed tutorials that clearly explain the best way to deploy, use, and manage Cloudera products. Login or register below to access all Cloudera tutorials.

Cloudera named a leader in 2022 Gartner® Magic Quadrant™ for Cloud Database Management Systems Get the report
    • Cloudera Cloudera
    • X
    Call us at(888) 789-1488
    International:+1 (650) 362-0488
    Email Sales
    Support Portal

    Ready to Get Started?

     

    NOTICE

     

    As of January 31, 2021, this tutorial references legacy products that no longer represent Cloudera’s current product offerings.

    Please visit recommended tutorials:

     

    Introduction

    Now that we know how to develop a Storm topology, let's go over how to package it up into a JAR file and deploy it onto a cluster.

    Outline

    Packaging a JAR

    In a terminal, navigate to the trucking-iot-demo-storm-on-scala directory and run:

    scripts/rebuild-and-deploy-topology.sh

    After installing some dependencies, this script runs the command sbt assembly under the hood in order to produce an uber jar, housing your topology and all of the dependencies. The jar is saved to target/scala-2.12/trucking-iot-demo-storm-on-scala-assembly-1.1.0.jar.

    Deploying to Storm

    If you ran the command in the previous section, then the topology was build and then also deployed.

    scripts/rebuild-and-deploy-topology.sh

    Feel free to open that script to see what it is doing. Notice that under the hood, it runs a command that looks like the following.

    storm jar trucking-iot-demo-storm-on-scala/target/scala-2.12/trucking-iot-demo-storm-on-scala-assembly-1.1.0.jar com.orendainx.trucking.storm.topologies.KafkaToKafka

    storm will submit the jar to the cluster. After uploading the jar, storm calls the main function of the class we specified (com.orendainx.trucking.storm.topologies.KafkaToKafka), which deploys the topology by way of the StormSubmitter class.

    Summary

    Congratulations! You now know about the role that Storm plays in a real-time data pipeline and how to create and deploy a topology from scratch.



    Your form submission has failed.

    This may have been caused by one of the following:

    • Your request timed out
    • A plugin/browser extension blocked the submission. If you have an ad blocking plugin please disable it and close this message to reload the page.