“Cambrian explosion” of data costs banks dear

A Cambrian era of data is upon us, according to Tabb

The rising volume of market data is comparable to the Cambrian explosion in which the number of species on Earth dramatically increased around 542 million years ago. Banks will have to adjust to this deluge of data if they are to keep pace with aggressive new competitors.

A new report from Tabb Group  estimates that global spending on market data was about $25 billion in 2013, with another $3.1 billion spent on market data infrastructure. Within those figures, $12 billion was spent on real time market data. According to Tabb, exchanges and other trading venue data feeds generated $2 billion in revenue for the exchanges. Aggregators and other intermediaries then turned that data into an addition $14 billion.

While Tabb did not invent the analogy with the Cambrian Age, the report suggests that it is a “vivid and appropriate” metaphor for market data at present, with volume, velocity and variability of market data placing increasing pressure on financial institutions to deliver improved performance. Banks are increasing their data by terabytes per day, according to Tabb. Unfortunately, the firm also notes that innovation is accelerating at about the same pace as the underlying data changes, making it difficult to keep up.

“Your market data tools either enhance or handicap your ability to compete effectively,” said Paul Rowady, senior analyst at Tabb Group. “Innovations in market data technology need to be simpler and less costly while providing increasingly diverse explosion of output.”

The theme of doing less with more which dominates the Tabb report is unlikely to come as a surprise to many. Banks and brokers have seen commissions and trading volumes decrease in recent years, while regulatory pressures have increased. However, it’s not all bad news. Tabb does estimate that managed services such as cloud-based infrastructure for market data can save up to 75-90% in some cases, making it one of the most impactful tools available to keep down the costs of market data.

Getting rid of inefficiencies such as expensive infrastructure, investments in data that is never used and long lead times of app development are part of the appeal of cloud, and Tabb notes that financial technology start-ups and capital markets firms that don’t have high performance market data needs have been keen to adopt cloud solutions.

Further progress can be made by paying more attention to how intelligence can be gained from data. More data from new sources and new ways of looking at data are on the rise, with exchanges continuing to explore this area. Hand held devices and crowdsourcing platforms as well as social media are all highlighted by Tabb as contributors to the big data phenomenon. Regulatory action and increased demands for transparency have also contributed.

Financial institutions will need to have trading strategies, risk analytics and plans that are more multi-asset, multi-regional and multi-temporal than previously, due to the increasingly joined up world of trading and the rise of more sophisticated technologies and opportunities. Tabb warns that there is also a need to use third party and bespoke applications, including order management, trade execution, risk analytics and other workflow components, to add to the supply of data.

The final point in the report is that more granular pricing models are necessary for market data. Tabb points out that as banks downsize, they could be experiencing waste as existing contracts with data vendors may provide data that is no longer used, and may continue to be unused for months at a time as the original individuals and teams associated with the data no longer exist at that firm. Tabb suggests that usage pattern monitoring should be used to make sure expensive data isn’t being wasted.

“The only way you’ll be able to compete effectively on the road ahead is if you learn to harvest more intelligence from more data,” said Rowady. “Infrastructure needs to be simplified as much as possible. Leverage managed services and cloud offerings for as many use cases as possible. The chances are the future has more in store for us than we have ever seen in the past. The need for innovations in market data technology has never been greater.”

For those interested, the Cambrian explosion is still hotly debated by scientists, who argue over whether there were precursors to many of the species that appeared at that time. Among the organisms preserved in fossils from this time are relatives of crustaceans and starfish, sponges, molluscs and algae, as well as the first ancestors of many other types of fish and land animals that appeared millenia later.