Intercontinental Exchange (ICE) operates the leading network of global futures, equity, and equity options exchanges as well as global clearing and data services across financial and commodity markets. The NYSE is the world leader in capital raising, listings, and equities trading.

The exchange processes and generates massive streams of data, which it leverages to better meet ever changing market and customer demands. However, as its data lake grew, legacy data technologies struggled to scale, forcing staff to silo data.

The Cloudera platform has helped the exchange gain real-time insights across more than 20 PB of data to improve market surveillance and confirm member compliance. The platform also enables the organization to incorporate new data sources and extend its analytic use cases to include machine learning.

According to Steve Hirsch, Chief Data Officer of Intercontinental Exchange, Cloudera’s enterprise data hub solved the organization’s data challenges. The new enterprise data hub supports data processing and analytics across more than 20 PB of data, with 30 TB of fresh data added daily.

Cloudera's flexibility to support on prem and off prem or cloud workloads, leadership solutions, and expertise in open source technologies were important to the exchange.

“We leverage Cloudera Navigator to provide us with key governance capabilities as well as visibility into our data lineage,” said Hirsch. “Cloudera's leadership with Apache Spark has enabled us to take advantage of the key improvements and capabilities of this ever-changing technology. And, Cloudera  has replaced our legacy database ad hoc query platform with Apache impala, enabling our user community to use SQL to analyze the contents of our data lake.”