Agility PR: Big data platform amplifying media coverage in real-time and in front of the people that matter

Agility PR Solutions streamlines media monitoring, outreach, and media intelligence in one intuitive platform for public relations professionals. As a company that has always dealt with the relentless barrage of the daily news cycle, one of the main technical challenges has been coping efficiently with large amounts of incoming data - importing it, storing it, and making it available to clients.

Agility has a unique implementation where its customers are the consumers of data, rather than internal users, and data is preserved for a long time for searching and reporting purposes. Searches are periodic, but the queries can be complicated. As a result, both the amount of data and the speed of processing are key considerations when evaluating solutions.  The partnership with Cloudera, formed several years ago, has enabled Agility to drive their big data strategy forward.

Challenge

When Agility started, big data was not an option. At the time the business was reliant on MySQL as a storage backbone. It didn't take long to find the limits of this setup, and Agility began to look for emerging technologies to augment its technical solution. Some projects existed that allowed the provision of fast search indexes that were stored outside of MySQL, but these were fringe technologies that depended on extremely expensive hardware to run.

With the emergence of social media - especially Twitter - as a vital component in true media monitoring, as well as an ever-growing client base, the amount of data Agility needed to consume started to grow dramatically.

Hadoop was emerging as the new technology to handle this huge paradigm shift in data availability. The Agility team was attracted to distributed computation models and began to devote serious R&D time to researching the technology and how to apply it. At the vanguard of this exciting evolution of data processing, Cloudera had begun offering courses and they were focused on serving the needs of business clients, so the fit was clearly aligned.

Solution

As a result of the training received, Agility began to use Cloudera Manager as part of the wider Cloudera platform. Its HDFS NameNode HA feature alone showed that Cloudera understood how to address vital issues to ensure system resilience and business continuity, and in doing so, could attract more people to the technology.

All of Agility’s interactions with Cloudera demonstrated they were not just interested in where big data might go, but absolutely invested in a new paradigm of data-driven business and being at the forefront of emerging companies that would help develop and contribute to open-source projects. They also provided real value through their own tools, driving the evolution of distributed big data technology.

Shortly after starting its journey, it became clear that Agility would need to invest in internal infrastructure to cope with the new realities brought about by the sheer volume of available data. Cloudera’s Professional Services team spent considerable time on this critical project. “They took the time to understand Agility’s business model, use case, and projections for growth. The help and guidance they gave in designing the physical cluster was invaluable,” said Diane Vuignier, VP of Product Development.

Results

In addition to the technology advantages and expertise the Cloudera platform offered, Agility realized significant cost savings. Using Cloudera Manager reduced operational costs of Agility’s big data cluster by 70%. The solution simplified setup, configuration, and regular maintenance with an easy user interface. Health stats ensured clusters were running optimally, and if there were any issues, Cloudera was there to troubleshoot. “Wrangling separate clusters is difficult to do by yourself, and Cloudera’s service effectively eliminated the need for hiring an in-house employee dedicated to this function. Additionally, outside of initial training, the advice Cloudera provided to Agility was all included, eliminating the need for expensive services that other vendors charge,” added Vuignier.

Now, companies stand again at the beginning of a new era of technology, with cloud and hybrid solutions being available paths forward. Cloudera is there once more to help steer Agility into its next technological evolution. There is no doubt that the ongoing partnership with Cloudera will continue to be a valued part of Agility’s success. 

With Cloudera Data Platform (CDP), Agility can visualise its journey to the cloud with more clarity and look forward to utilizing aspects of the cloud to evolve a more stable, streamlined and performant cluster.

“CDP Operational Database alone offers management of vital services like HBase, and utilises S3 as a data lake to remove a whole category of hardware considerations that free us up to apply our resources to more exciting things like new features,” added Vuignier.

The company chose Cloudera for its Big Data platform to invest in a new paradigm of data-driven business, and to be at the forefront of emerging companies.  Looking back over the 8 years that the business has been associated with Cloudera products and support, the answer is clear for Agility PR Solutions: “There's never been a reason to consider anyone else,” Vuignier concluded.