Real-Time Agentic DataOps

The demo walks through Ratatoskr's React dashboard, starting with an Overview page that shows live system health, Flink cluster status, and recent jobs. From there it moves through the Agent Catalog of registered agents, the Agent Designer canvas for building agents visually, and the Agentic Studio, where agents are chained into pipelines and run, with results tracked on the Runs page. It ends on the Settings page, where users connect an LLM such as Cloudera AI Inference to power the ReAct agents.   Watch the demo

• Ratatoskr starts a complete Dockerized Flink and Kafka stack with a single CLI command and manages it through a FastAPI Control API.

• Its Agent Designer lets developers build workflow or ReAct agents on a visual canvas and compiles them into runnable Python and Flink YAML.

• The Agentic Studio chains multiple agents into streaming pipelines that run either locally or on the Flink cluster.

• Deterministic agents monitor NiFi and Kafka, correlate incidents across both systems, and only fix problems within gated phases or after a human approves.

• ReAct agents use Cloudera AI Inference to enrich security alerts and write incident briefs or debug runbooks without changing any live system.

1. Honeypot / cybersecurity

2. NiFi flow monitoring

3. Kafka cluster monitoring

4. Cloudera manager monitoring (CDP)

5. Cross-signal correlation

Ratatoskr has three layers on top of Apache Flink. At the top is a control plane: the Ratatoskr CLI and a React dashboard both talk to a FastAPI Control API on port 8090. That API manages the agent registry (two YAML files, one for the dashboard catalog and one for runtime entry points) and turns the Agent Designer's canvas output into Python, Flink YAML, and local runners. Below that is the runtime, which runs in Docker Compose: a Flink JobManager and TaskManager plus a dedicated "Studio" Kafka broker that moves events between pipeline stages. The agents themselves run as Flink jobs and come in two kinds. Workflow agents are deterministic and handle monitoring, thresholds, and event correlation. ReAct agents call an LLM, optionally Cloudera AI Inference, to enrich alerts or write runbooks. The Agentic Studio chains these agents into linear Source → window → Agent(s) → Sink pipelines that run either locally or on the cluster. Optional profiles add a Cowrie honeypot or an Apache NiFi instance as real event sources. The core design choice is that detection and correlation stay deterministic, LLMs only explain, and any change to a live system has to pass a gated phase or human approval first.