Your browser is out of date!

Update your browser to view this website correctly. Update my browser now

×

The Financial Industry Regulatory Authority, or FINRA, is an independent regulator overseen by the Securities and Exchange Commission that is obligated to monitor the markets it operates. FINRA oversees every brokerage firm and broker doing business with the U.S. public and monitors trading on the U.S. stock markets.

 

Read the case study

Overview

The Financial Industry Regulatory Authority, or FINRA, is an independent regulator overseen by the Securities and Exchange Commission that is obligated to monitor the markets it operates. FINRA oversees every brokerage firm and broker doing business with the U.S. public and monitors trading on the U.S. stock markets.

FINRA finds evidence of market manipulation by assembling 50 billion market events into a holistic picture of the U.S. securities market every day. By rewriting this process using Cloudera and other Big Data technologies, FINRA is creating a regulatory platform that will handle future market growth while containing escalating costs and increasing operational efficiency.

Solution

Every day, FINRA receives data feeds containing orders, quotes, and trades from securities firms and various exchanges. Since transactions may span several data providers over days or weeks, understanding order lifecycles requires that huge volumes of data be linked across these different feeds and updated as new information is received. This requires very rapid data access, which was provided by implementing Apache HBase as part of FINRA’s CDH deployment.

A Hadoop architecture allows FINRA to use a public cloud infrastructure, in this case provided by Amazon Web Services. This cloud-based implementation delivers the elasticity, cost-savings, and enterprise-grade infrastructure support that FINRA needs.

Impact: Increased Performance and Operational Efficiencies

FINRA's program to implement market regulation applications using Big Data and cloud technologies including Cloudera is containing escalating platform costs and improving operational efficiency. This frees resources to focus on improving systems to provide greater flexibility and effectiveness for regulatory analysts in several key ways.

  • FINRA’s Hadoop-based platform allows the use of less expensive, industry standard hardware.
  • The Hadoop-based architecture lends itself to deployment on a public cloud such as Amazon Web Services, allowing FINRA to benefit from the operational economies of scale of such a vender, as well as providing the elasticity to avoid over-provisioning to handle peak loads.
  • The legacy environment was also challenged with capacity limitations, causing FINRA to spend much effort and analysis determining where and when available capacity could be used for various types of analytics. By removing the capacity limitations, the Hadoop/cloud combination virtually eliminates this effort.
  • Cloudera Manager automatically manages extremely complex operational tasks within Hadoop, allowing FINRA to focus resources and expertise on areas of strategic regulatory importance rather than fundamental IT tasks. This saves costs while increasing FINRA's ability to accomplish core objectives.

Meanwhile, the use of HBase to access the order lifecycle graph database has reduced response times for certain complex queries by orders of magnitude. One complex query in particular that took ninety minutes to run was reduced to ten seconds. This allows analysts to rapidly iterate and quickly converge on answers that would have been prohibitive in the prior system.