Financial services IT departments are experiencing quantum leaps in data volume and application complexity that require commensurate quantum leaps in underlying financial services technology. The cloud in all its flavors: SaaS, PaaS and IaaS, public and private, offers the most promising refuge.
The Financial Market Data Explosion
Eric Schmidt, CEO of Google, recently claimed that “every two days we create as much information as we did up to 2003″. While his comment was largely directed at consumer media and the creation of user-generated content in applications like Facebook and Twitter, it is equally if not more relevant to financial market information. Like consumer media, financial markets generate reams of unstructured data from earnings call transcripts to analysts reports to news to blog posts. However, financial markets are also generating exponentially increasing amounts of market data due to high frequency trading, venue fragmentation, and an ever expanding pool of financial instruments.
High frequency trading, market fragmentation and instrument proliferation
are driving exponential increases in the volume and variability
of real-time and historical financial market data
Financial services IT shops are struggling to keep pace with this explosive growth in market data. Moreover, the financial market data explosion isn’t limited to real-time trade data, because every tick flows into gargantuan historical databases used for modeling and analysis. The volumes are simply staggering and the resulting pressures on financial services IT infrastructure predictable. Fully two-thirds of financial services firms fear their analytics programs and infrastructures will not be able to handle increasing analytical complexity and data volume, according to a just-released research report, featuring a survey of financial services IT professionals conducted by Wall Street & Technology in conjunction with Platform Computing, SAS and The TABB Group.
Attention Economics and The App Revolution
Translating these exponentially increasing amounts of data into useful information is no easy task. Just like consumers, business users are constantly making trade-offs in how much time and attention they are willing to spend digging through irrelevant data to get to the relevant information they need to do their jobs. Business users need more targeted access and specialized applications. This requires financial applications to shoulder an ever increasing burden of scrubbing, filtering, crunching, and presenting the data in bite-sized chunks of useful information that the average human being can digest and use to take positive action. However, the simpler it gets for the user; the more complex it gets for IT.
The everything-in-a-box Bloomberg terminal model is breaking down, because it places too much of this increasing data burden on the user and not enough of it on the application. Financial services IT departments are no longer willing to pay for irrelevant data just because it comes “in-the-box,” and are increasingly opting for specialized and mobile apps that provide users and downstream applications with just the market data they need, where they need it, when they need it.
Competitive Advantage and the Need for Speed
Only an order-of-magnitude increase in agility will enable financial services IT departments to handle these order-of-magnitude increases in demand for data and applications. High frequency trading isn’t the only area of financial services IT where speed is critical. Sustainable competitive advantage is built on the ability to deploy and evolve critical IT services and infrastructure faster than the competition, not on some unique algorithm or application. Inflexible architectural and deployment models that rely on the false assumptions of stable user requirements and a stable industry environment are feeling the crush between the exponential demand for data capacity on one end and the increasing demand for specialized applications on the other end. There just isn’t enough time to keep up.
While financial services IT professionals will no doubt prove to be the most cautious adopters of cloud technologies, they are perhaps in the most need of them. The cloud in all its flavors: SaaS, PaaS and IaaS, public and private, offers the most promising refuge from this crisis. The reason is straightforward: on-demand. To quote Wikipedia: Cloud computing…resources, software, and information are provided to computers and other devices “on demand, like the electricity grid“.” From on-demand deployment of private cloud server instances to handle real-time risk management processing spikes to instant upgrade of a SaaS portfolio management application, cloud solutions shrink deployment and maintenance time by an order-of-magnitude. The speed gains offer the flexibility financial services IT departments need to adapt to the increasing and variable demand for data and applications, while the time savings can free up overburdened IT resources to focus on projects that deliver the most competitive advantage.
[...] While many people have heard of automated stock trading and some of its more spectacular consequences like the 2010 Flash Crash, what most people don’t know is that there is a financial information technology arms race going on that has produced radical changes in stock trading over the last ten years. With every progression in industry standards, increase in processor speed and decrease data storage costs, automated traders up the ante by developing faster, more computationally intensive algorithms to outwit the market and outsmart each other. The result is a feedback loop that demands ever increasing data processing power and data management efficiency. One can debate whether this is a good thing or a bad thing, but what is not up for debate is that financial services firms that want to stay competitive must constantly look to new technologies to avoid being crushed under the relentless deluge of stock trade data. [...]
[...] these trends add up to the the exchange version of the “too much data, too many apps crush” that is being felt throughout the financial services sector. In response, The Melbourne [...]
[...] these trends add up to the the exchange version of the “too much data, too many apps crush” that is being felt throughout the financial services sector. In response, The Melbourne [...]