ClouderaNOW Activate your data, unleash your AI, and scale across any cloud | Dec 9

Register
  • Cloudera Cloudera
    • Blueprint Card

    Overview

    Simplify enterprise AI deployment and optimize infrastructure costs with scalable, shared AI services on Cloudera.

    As organizations expand their use of AI, deploying and managing separate infrastructure for every model, application, and intelligent agent becomes increasingly expensive and operationally complex. Dedicated environments can lead to underutilized computing resources, fragmented management, and slower application delivery.

    The Microservices with Ray Blueprint combines Cloudera AI with distributed computing to provide a unified environment for deploying, scaling, and managing AI models and services. By sharing CPU and GPU resources across multiple workloads, organizations can improve infrastructure utilization, respond to changing demand, and reduce the operational effort required to support enterprise AI applications. With centralized monitoring and automated resource management, teams can accelerate AI delivery while maintaining greater control over infrastructure costs and performance.

    Business Value:

    Reduce the computing cost and complexity of enterprise AI with Ray on Cloudera, improving CPU and GPU utilization, accelerating model deployment, and scaling AI applications efficiently through shared infrastructure, automated resource management, and centralized operational visibility.

    • Cost Savings - Reduce AI infrastructure costs by consolidating multiple models and services onto shared CPU and GPU resources; Improve hardware utilization through intelligent workload placement and dynamic resource allocation; Lower AI operations costs through centralized deployment, monitoring, and management; Minimize unnecessary compute spending by automatically scaling workloads and retiring eligible idle resources

    • Cost Avoidance - Avoid overprovisioning expensive GPU infrastructure through flexible capacity management and resource sharing; Reduce operational risks associated with fragmented AI deployments and inconsistent infrastructure configurations; Minimize service disruptions through integrated performance monitoring and workload health visibility; Avoid redundant infrastructure investments by supporting multiple AI models, inference engines, and agent services within a unified environment

    • Revenue Growth - Accelerate time to market for AI-powered applications through standardized, reusable deployment workflows; Increase engineering capacity to deliver new AI services without proportional increases in infrastructure management effort; Enable new AI-driven products and customer experiences through scalable model inference and agent services; Support growing AI adoption and application demand through flexible infrastructure that expands with business needs

    Demo

    This demonstration showcases how organizations can use Microservices with Ray on Cloudera AI to simplify AI model deployment and manage shared computing infrastructure.

    See how a ready-to-use environment establishes a distributed AI cluster, allowing teams to deploy a language model and make it available to applications through a standardized interface. Explore how computing resources are allocated across the cluster and how the platform provides visibility into running models, available capacity, and operational health.

    The demonstration also highlights centralized monitoring and resource management, illustrating how organizations can support multiple AI workloads, scale resources to meet changing demand, and reduce the complexity of operating enterprise AI infrastructure.  Watch the demo

    Key Features

    • Unified AI Service Management: Deploy, manage, and monitor language models, AI services, and agent tools through a centralized interface, reducing the complexity of managing separate AI environments.

    • Shared Infrastructure Utilization: Run multiple AI workloads across common CPU and GPU resources, helping organizations reduce idle capacity and improve infrastructure efficiency.

    • Intelligent Resource Scaling: Adjust model capacity and provision additional computing resources in response to changing demand, improving operational flexibility.

    • Flexible AI Model Deployment: Support multiple language model serving technologies and AI services within the same environment, giving teams greater flexibility in model and application selection.

    • Distributed AI Processing: Support larger AI models by distributing workloads across multiple GPUs and compute nodes, expanding the range of models organizations can deploy.

    • Automated Resource Management: Configure infrastructure limits and retire eligible idle resources to help control operating costs and prevent unnecessary resource consumption.

    • Centralized Performance Monitoring: Track infrastructure health, model performance, and resource utilization through integrated operational dashboards.

    • Accelerated AI Deployment: Get started with a reusable Cloudera AI deployment template that simplifies setup and provides a standardized foundation for scaling AI applications.

    Use Case

    Organizations scaling enterprise AI often struggle with rising infrastructure costs, underutilized computing resources, and the operational complexity of managing separate environments for different models and applications. Traditional deployment approaches can require dedicated infrastructure for individual workloads, making it difficult to adapt capacity as demand changes and slowing the delivery of new AI capabilities. The Microservices with Ray Blueprint addresses these challenges by combining Cloudera AI with distributed workload management to deploy, scale, and monitor multiple AI models and services across shared computing resources. By consolidating AI infrastructure and automating capacity management, organizations can improve resource utilization, reduce operational overhead, accelerate application delivery, and support growing AI adoption more efficiently.

    Architecture

    A CAI application runs the Ray head, nginx ingress, the management API and the worker-capacity controller. Additional CAI application pods join as CPU or GPU Ray workers. Ray Serve schedules model deployments; tensor-parallel replicas reserve complete placement groups with one GPU per executor bundle. Engine virtual environments and durable project state reside on shared NFS. Ray Serve controls replica count, while the capacity controller evaluates unmet Ray demand and calls the CAI API for missing workers. Empty-worker retirement uses a rejectable Ray idle drain before deleting the owned CAI application. Prometheus and Grafana run as separate CAI applications; Grafana access is proxied through the authenticated head origin. Cross-pod collectives require the documented Istio configuration.

    Get started by deploying this blueprint in your environment

    Every blueprint ships with source code, sample data, and step-by-step instructions — ready to run on your Cloudera Platform

    Your form submission has failed.

    This may have been caused by one of the following:

    • Your request timed out
    • A plugin/browser extension blocked the submission. If you have an ad blocking plugin please disable it and close this message to reload the page.