Index Exchange: Optimizing advertising campaigns through data analytics

Index is a Supply Side Platform (SSP) and leverages programmatic advertising, a type of technology that increases efficiency and allows media buyers to run advertising campaigns at scale. Data is crucial to an SSP’s business so that advertisers can tailor ads to a consumer's preferences, delivering ads that are meaningful to the consumer.

Challenge

Brands and marketers partner with a Demand Side Platform (DSP) while publishers looking to sell their ad space inventory partner with an SSP, triggering an automated auction. Programmatic technology makes this matching process easier, which ultimately means better ads for consumers and better value for brands and marketers.

The amount of data and bid requests the exchange processes is growing at an exponential rate due to increased web traffic and the emergence of connected TV (CTV) and mobile applications. The exchange leverages the data for reporting and analyzing the performance of advertising campaigns, with a focus on real-time visibility, optimization, and complete control over the campaigns. This data needs to be available in a timely manner to ensure the exchange is working properly and that the auctions are as efficient as possible. It is updated on an hourly cadence, where analysts monitor trends and make adjustments on the fly.

Index Exchange has seen a 5x growth in volume, from 50 billion to roughly 250 billion auctions per day within the past few years, resulting in 150 TB of data being processed daily, and needed an efficient and scalable way to manage and process data.

Solution

Index turned to Cloudera’s platform for data warehousing and daily operations in order to keep up with the volume of data and complexity of analytics needed. There are 500 nodes currently in production, ingesting data from 8 data centers across different continents.

“We’ve built Data Platforms 1.0 and 2.0 and are now exploring Data Platform 3.0. Choosing Cloudera made things easier with one unified platform with lots of features. The platform performs and scales well, with all the toolsets and management capabilities that we were looking for, it’s easy for our engineers to manage and was a cost effective option for us,” said Brian Hu, Director of Data Engineering at Index.

Scalability was a big factor in the decision, as the exchange expects enormous volume over the next 5-10 years. “We needed the ability to quickly scale up and have consistent performance. Our company moves quickly, so this has had a huge impact for the team,” added Hu.

Index is still currently in the process of building out their data platform foundation further, and will add additional data types and use cases in the next phase. Today all workloads remain on-premises, as the company already owns and operates a massive infrastructure with over 10,000 servers on the edge around the globe. Choosing Cloudera gives the company the flexibility to have a hybrid architecture, keeping workloads on-premises and utilizing cloud solutions for other use cases, if desired.

Results

The new Cloudera enterprise platform has given Index the ability to quickly scale up, adding services to the cluster and getting data to people much more quickly. It is a critical component of the exchange and is invaluable to the customer experience. Data is now provided on an hourly cadence so that internal users are able to monitor monetization and traffic sources more closely. Customers have more insight into what is going on with their campaigns, and can make adjustments when necessary. Analysts look at trends with the different traffic sources, investigate anomalies and reach out to customers to resolve any issues on a daily basis. Without the data that the platform provides, this process would take days instead of hours, which would result in a massive financial impact. Through this timely data that the Cloudera platform provides, Index is able to save its customers money by showing them where adjustments need to be made to further optimize their campaigns.

Over the longer-term, Index will continue to leverage machine learning and data science use cases to go deeper into the data it has. This will enable a more proactive and aggressive approach in getting value from data, leading to more efficiency for the exchange.