Partnering with Atos to build a Big Data appliance

Blog Data Science Financial Services Technology

We are proud to announce our new partnership with Atos to build a Big Data appliance aimed at serving banks and other financial institutions.

The appliance combines ActivePivot with a bullion server from Atos as well as the Zing Java Virtual Machine from Azul Systems.

Bullion is an innovative, modular server that can scale up (or down) from 2 CPUs and 1.5 TB of memory to 16 CPUs and 12 TB. It fits well with the equally scalable nature of ActivePivot, which fits just as well on smaller machines as on the 16 TB juggernauts, as we demonstrated last year.

The Zing JVM enables the appliance to go beyond the usual limitations of Java, , allowing heap sizes of hundreds of GBs. It uniquely complements ActivePivot’s top-of-the-line management of off-heap memory and enables the appliance to work without pauses on data sets of several TB.

We designed this Big Data appliance together with our partners to anticipate the needs of financial institutions related to the new FRTB regulations. When discussing the matter with our customers, it became obvious that one of the major changes brought by FRTB would be a dramatic increase in the volume of data that financial institutions would have to process every day, mainly due to two new requirements:

  • The transition from VaR to Expected Shortfall as the risk metrics of reference. Calculating stressed Expected Shortfall requires a much higher level of storage and computing power as historical observations have to be stored for comparison between the current position and a similar stressful position in recent history. Thus observational data needs to be stored from at least 2005.
  • P&L attribution with multiple liquidity horizons requires an exponential amount of data to be stored and processed

In-memory analytics is already the best technology to analyse large data sets efficiently. Coupled with a hardware that has a memory pool of up to 12 TB, it is able to meet the needs of the largest institutions while the scalable nature of both bullion and ActivePivot offers a close control of hardware costs and the ability to ramp up gradually during a transitional period.

The appliance also offers an excellent synergy with ActiveStorage, our persistent storage solution designed to maximize data loading speeds. ActiveStorage is capable of loading a full 12 TB in less than half an hour, making it the solution of choice to maintain reactivity and interactivity when performing analytics on vast volumes of historical data, like P&L vectors.