The Gr8 Conference: Grid computing and computational finance

Jonathan Felch is clearly an amazingly smart developer who has been using Groovy and Grails to solve some really interesting problems in the field of quantitative finance.

The most common languages in quant finance (which I know nothing about, so apologies in advance for any mistakes in this write up!) are C++ and VBA. Unfortunately it’s hard to write large, scalable applications in VBA, and in a world where new ideas need to be implemented very quickly and the vast majority of them are of no value, C++ simply isn’t fast or cost effective enough for trying out new ideas. Jonathan reviewed a number of different scripting technologies on the JVM and even the possibility of leaving the JVM, but in the end Groovy was chosen as the JVM is a great ecosystem for building enterprise applications, and Groovy integrates more deeply and thoroughly into the JVM (and specifically plays better with existing Java code) than other scripting languages like JRuby, Jython, Scala or Clojure.

The core objective of the application was to develop a framework that would allow for the quick development and analysis of potential quantitive trading strategies. For example, perhaps if a company has a large debt load with a small bond payment due followed by a much larger one, the risk of the company defaulting on the small payment might be overestimated using traditional models, providing buying opportunity for traders who could then pay a little more for those small bonds because they found out from historic data that the defaults on the initial bonds are lower than would be expected by the overall debt load of the company.

Jonathan and his team have to put together a system that would allow for quickly implementing reports that would show whether such hunches fit with historic data and whether it might make sense to use them as trading strategies for the bank.

There is a lot of essential complexity within the domain. With different calendars, different measures and a lack of truly unique identifiers (for example, bonds can have multiple ISINs over their lifecycle), there are a number of problems to be solved just to provide clean data for analysis. There is also a problem of storing and working with the data as there is a huge amount of information to analyze.

The system they developed was comprised of a Java core with a real time credit market gateway, a real time Bloomberg gateway for asynchronous response/requests, and EJB3 stateless session objects for accessing internal bank and third party databases. On top of that they built a Groovy/Terracotta data cache containing both the logic for calculating various reports, and a cache for storing the results of previous calculations. They also developed a Grails reporting front end with and HTML interface, JSON web services, Excel interfaces and a JMS gateway.

The interesting part of the system was in the data cache which provided chainable finders/builders so that if there was a cache miss, scriptable math within the builders would be fired off to pull and calculate the appropriate values. They also added a dependency graph for the calculated data to identify any results that had been invalidated by updated data or calculations.

One of the initial concerns had been whether Groovy (or any scripting language) would perform well enough for this kind of calculation intensive application. In the end, the experience was that the programming approach and algorithms had much more effect on the performance than did the underlying language and by using a smart caching strategy they were able to meet their performance goals in Groovy.

Unlike a traditional data model with all of the data represented in the database and (often) accessed through Hibernate, their system used tuples, with a wide variety of data sources (not just databases but also data feeds) being used to provide information and a combination of stored and calculated data being contained within the cache. Everything was decomposed into tuples that could implement map and list, allowing efficient operations in Terracotta.

It was great to see a real world case study of the use of Groovy and Grails – especially given the fact that it was an application where performance was so important. It was also a really good introduction to the way that algorithms are trending for dealing with large sets of data (I know map/reduce isn’t exactly new, but it was still interesting to get a glimpse of some of the architectural patterns they had used).

The session did spend a lot of time on the domain and perhaps it would have fitted better a little later in the conference after we’d been satiated by Graeme and Guillaume providing us the details of the new features in Grails 1.1(.1) and Groovy 1.6(.3) respectively, but it was definitely a thought provoking session that challenged a lot of the traditional thinking about the limitations of scripting for data and calculation intensive applications and that did a great job of showing off the benefits of Groovy and Grails for developing business applications on the JVM.