Financial institutions have long used credit valuation adjustment (CVA) to monitor and manage counterparty credit risk, meet regulatory and reporting requirements, and even price and hedge CVA. In the post-2008 credit crisis world, however, this evaluation activity has steadily increased in demand and complexity, creating tough computational challenges for firms.
Where once large banks used CVA on a monthly basis, they’re now performing these calculations at daily to real-time (sub-second) intervals. The active, continuous management demanded by counterparty credit risk (CCR) raises two computational problems in particular: latency sensitivity and vast scale. Alone, these two would challenge traditional scale-out grid infrastructures. Combine them with other routine risk workloads as well as the U.S. Federal Reserve’s CCAR stress tests, and the growth in required core count is eye-popping.
So what’s happening here exactly?
Financial firms use Monte Carlo simulations to price and measure CCR and view probabilistic results of any credit decision. This is a very powerful tool — but the challenge is in performing the simulations quickly enough to make a difference.
When performing whole portfolio CVA, for example, the complexity and scale of data can be too large for the memory on any one node. The calculation must partition across multiple cores and requires results calculated on different nodes. This process decreases throughput in a typical “scale out” grid architecture, while the increased core counts creates connectivity and latency bottlenecks.
The result? Scale-out architectures don’t scale efficiently enough for users needing rapid results. Firms might also try a scale-up approach by adding more cores to a single compute node. But again, they’re turned back by increased latency, inter-system communication … and cost.
How Cray approaches this with customers
The Cray® XC™ series supercomputer delivers a ready solution to the computational challenges of CVA. How? Let’s look at two Cray technologies that are key to solving these tenacious scaling and latency issues while also keeping costs under control.
The primary design criterion of the Cray XC system is sustained performance. We achieve that performance in no small part because of the Cray Aries™ interconnect technology. Aries is packet-switched with adaptive routing to avoid network congestion and failures. The result is a cost-effective, scalable, high performance global bandwidth resilient to workload variations and job placement. Based on a Dragonfly network topology, the Aries-based network addresses the performance and latency issues of more traditional “fat tree” approaches by incorporating a three-tier design for interconnecting nodes: an electrical backplane, copper cables and optical fiber. Compared to traditional fat tree InfiniBand networks, the Aries interconnect and Dragonfly topology provide a two-to-one advantage. Each packet takes one optical hop at most.
The second technology is the Cray® DataWarp™ applications I/O accelerator. DataWarp is a shared burst buffer filespace on specially designed I/O blades and connected to compute nodes via the Aries interconnect. What it means for CVA is the release of data access bottlenecks. If CVA calculations that need to pass data between nodes are too large to run in a single node’s memory, they must use a shared resource. The DataWarp technology brings CVA I/O closer to compute resources by providing faster access to shared data contained on the DataWarp infrastructure. Users can also skip the massive and slow initial condition broadcast of files to all nodes. Instead, all you do is send files to a handful of DataWarp blades, which can significantly reduce the wall clock time of a job.
Cray Reduces Grid TCO for CVA and all Monte Carlo Workloads
From a TCO perspective, these technologies contribute mightily as well. It’s important to realize that while CVA has particular computational challenges, using Cray technology gives you more options for addressing those challenges. But most importantly, a typical firm’s grid, whether used for risk, asset pricing, strategy backtesting, etc., can be made faster and cheaper using Cray’s technology — and who doesn’t want that?
For one, the embedded Aries network means no separate purchase, installation and support costs of additional inter-node networking equipment. More savings are found in the fact that you can configure the bandwidth of the Aries interconnect to match performance requirements and cost constraints.
DataWarp technology delivers similar cost efficiencies. As a result of the embedded and shared approach to providing fast SSD capability, files can be striped across DataWarp blades, which means you purchase blades based on your bandwidth requirements. You might only need a handful of DataWarp blades for a typical 1,000-node configuration. And rather than nodes having to be provisioned individually and statically with large SSD capacity, DataWarp can be provisioned dynamically and shared over Aries, saving significantly on the purchase and support of SSDs or hard disks on each node.
Going forward, managing risk, meeting regulatory requirements and staying competitive isn’t going to get less complicated. The Cray XC series supercomputer, armed with Aries and DataWarp technologies, is a clear answer to meeting those challenges while saving money.