BQuant Behind the Scenes: How Bloomberg Leveled the Playing Field for Quantitative Analysis in Finance

Back in 2013, Bloomberg’s world-class quantitative researchers were looking for an alternative to MATLAB, a programming and numeric computing platform for analyzing data, developing algorithms, and creating quantitative models. They needed something with additional capabilities, including interactive visualizations, compatibility with Bloomberg’s market data and the Terminal, and the ability to share results with colleagues or customers.

It turns out that many of our customers also needed just such a platform in order to remain competitive with the largest global financial firms that had built their own quantitative investing platforms. Today, quantitative researchers need so much more than a desktop solution.

Bloomberg recently introduced BQuant Enterprise, a cloud-based platform for quantitative analysts and data scientists in the financial markets. This customizable, turnkey solution accelerates financial services firms’ ability to compete more aggressively by enabling  customers to incorporate quantitative approaches throughout their investment processes. BQuant is the first off-the-shelf data science solution that is designed specifically for financial markets and that offers operation-ready access to Bloomberg’s comprehensive range of multi-asset-class financial and alternative data sets.

Let’s take a backstage look into the effort to bring BQuant from concept and vision to product launch. Along the way, we’ll learn about the contributions of academic researchers, leaders in the Python and Project Jupyter open source communities, application developers, and cloud architects.

Quants Prototype a Research Platform

Bloomberg’s Quant Research team incubates new ideas for customer products, internal tools, and infrastructure. It is often tapped to troubleshoot customer issues that can be addressed with machine learning or to pursue new approaches to quantitative analysis. “We reached the limits of what we could do with MATLAB,” said Bruno Dupire, Ph.D, Head of Quant Research in Bloomberg’s CTO Office. “We realized we had the tools and knowledge to build our own rich computational environment that could be used to explore financial analyses in multiple dimensions, simultaneously.”

The team envisioned building a computational layer for the Bloomberg Terminal — BQuant Desktop — with easy access to Bloomberg’s market-leading data. The Quant Research team decided to base its prototype on the Python programming language, which was becoming popular in the world of finance and had the advantage of being an open source ecosystem, rather than a proprietary language.

Around the same time, the company’s Desktop Build Group, which works directly with Bloomberg customers to build quantitative models and applications using Bloomberg data and services via the Desktop API, were working on a set of tools to make their job faster, thereby allowing them to focus on the financial problem rather than boilerplate code for developers. These tools were built to organize data with a query layer (now known as BQL), finance-specific libraries, visualizations, and bindings to commonly-used languages like .NET and Python. They saw an opportunity to empower a growing number of clients who were becoming more tech-savvy to become more self-sufficient by focusing on the business problem at hand.

“Collaborating closely with Bruno’s team, we built some really cutting-edge analytics and visualizations, using Excel as a UI, instead of as a calculator,” explained Javier Mitra-Valdes, Head of Product for BQuant.

The two teams collaborated to ensure BQuant was built with several important new capabilities. First, a user could write a Python function to access Bloomberg’s data sets in order to process them with best-in-class models and analytical tools. For example, Bloomberg’s Quant Research team used BQuant to look at an alternative data set of weather feeds to assess the impact of snowfall on U.S. retailers’ performance or cyclones’ impact on manufacturers.

Next, the team solved one of its toughest engineering challenges by creating bqplot, a highly interactive visualization tool that empowers users to dynamically input parameters and see the output get updated in real time. It offers widgets that facilitate the navigation through large data sets and offers compact representation of large collections of items, such as more than 1,000 stocks simultaneously. bqplot is now available as an open source tool in order to benefit a larger audience.

Finally, the entire notebook of algorithms, functions, data, visualizations, explanations, and recommendations can be stored together in a Jupyter notebook file and shared with other colleagues.

Python Developers Cultivate the Project Jupyter Community

The rise of Python was certainly a catalyst that made the BQuant platform possible. But the team needed more than a programming language — it needed a robust and growing ecosystem to support the development of something customers could use and share.

So, in 2014, Bloomberg started collaborating with the IPython (now Project Jupyter) community. The company recruited mathematicians who were actively expanding IPython notebooks and building ipywidgets and sponsored their open source contributions. Bloomberg also partnered in sponsoring the development of several tools and libraries, such as JupyterLab. In all these efforts, Bloomberg has purposefully sought to work with community leaders and contributors, such as the Project Jupyter Steering Council, in a healthy relationship that respects and promotes community governance and the inclusive, productive open source environment.

Bloomberg’s persistent investment of time and resources has helped improve the stability and resilience of Project Jupyter. This has required a delicate balance between keeping the features users know and love, while also making the platform’s code base more robust and structured. Project Jupyter now has a trusted community of more than 1,500 contributors who fuel functionality for millions of daily users at tens of thousands of scientific and industry organizations around the world.

Software Engineers Create User-Friendly Applications for Customers

With a prototype and software ecosystem in place, BQuant transitioned out of the incubation phase and over to product development teams who would take the concept through to commercialization. By this time in 2015, Bloomberg had recently established a new Engineering office in San Francisco — a region with a large talent pool of engineers with backgrounds in open source software and cloud infrastructure — and decided to base the BQuant Engineering team there.

“Our Bay Area location gave the BQuant teams a few key advantages,” explains Mario Cadete, Head of Bloomberg’s San Francisco Engineering office. “Being able to embed ourselves in this region’s diverse range of tech communities and draw from the rich pool of talent has enhanced our ability to think creatively, experiment, and adopt new approaches as we developed this innovative solution.”

The BQuant Engineering team has since grown significantly. Together, they are extending the features of Jupyter notebooks and interactive widgets, building libraries to support complex financial workflows, and scaling the system to work across both public and hybrid cloud infrastructures.

BQuant allows finance professionals to stick with the programming language they know best: Python. Behind the curtain, the apps and web interfaces are built using JavaScript, which makes the different analytics modules highly interactive, without users needing to learn a new language. The analytics help users overcome the limitations of Microsoft Excel, exploring more models based on greater volumes of data — all in less time and with reduced complexity.

One of BQuant’s most important features came out of a collaboration with Bloomberg’s Sales team and customers. Quantitative analysts could easily use notebooks to build increasingly sophisticated models and data visualizations, but the results needed to be used by portfolio managers and other non-technical, front-office colleagues. 

“Notebooks are incredibly valuable when our customers are performing research and generating investment ideas, but our sales reps and customers helped us understand we needed to provide an end-to-end solution enabling them to easily share their results with decision makers,” said Cheryl Quah, a BQuant Engineering Lead. Today, BQuant allows researchers to seamlessly publish their findings as an easy-to-consume, interactive application for their internal colleagues.

Cloud Architects Add Scalability and Security

Once BQuant Desktop was rolled out to Terminal customers, another team focused on creating a cloud-native version of the solution: BQuant Enterprise. It takes advantage of public cloud scalability to run complex machine learning over vast data sets and libraries in an automated way. By using distributed computing, users can run tasks overnight on a secure cluster to solve problems that are far beyond the capabilities of a desktop. In addition, BQuant Enterprise can ingest data from customers’ own data sets or ones they’ve sourced from third parties.

On the back-end, BQuant Enterprise uses Kubernetes container orchestration to manage the virtualization of the most recent version of Jupyter; Apache Spark’s unified analytics engine for big data processing; finance-specific open source libraries; NumPy’s mathematical functions; pandas’ data analysis tools; and more.

A different team focused on managing identities so users could interact with dynamic calculations from licensed data sets. When dealing with the volume and breadth of data Bloomberg offers, a lot of effort went into making the user entitlement process an invisible and seamless experience for customers, while ensuring data- and software-license compliance.

In addition, our engineers have gone to extreme lengths to secure each customer’s investment workflows and data. They’ve worked directly with the most popular cloud providers to segregate customers from each other, secure the environments, and meet compliance requirements.

Product Teams Bring BQuant Enterprise to Customers

We’ve just explored much of the innovative work it took to bring BQuant Enterprise to life: a new computational research platform with robust automation capabilities; a thriving Project Jupyter ecosystem; a range of applications and libraries for users; and cloud-native architecture. But what does BQuant Enterprise really bring to customers?

Bloomberg’s CTO Shawn Edwards regularly reminds our BQuant engineers that they need to make it incredibly easy for users — quantitative researchers, analysts and portfolio managers — to take an idea from exploration to production. “BQuant Enterprise enables clients to leverage the richest data sets, interconnect them, perform complex computations, and ultimately answer questions customers either haven’t been able to ask or that they have yet to think of. No one has solved this problem before now. We are making available practical, powerful machine learning that helps customers advance their investment models and algorithms to unlock new possibilities — and that’s what’s most exciting!”

“Bringing this product to market has been an incredible collaboration between multiple teams at Bloomberg — across Product, Engineering, CTO, Desktop Build Group, UX, Sales, and Support,” noted Mitra-Valdes. “To create and support a product of this nature — which is geared to a new type of Bloomberg customer — has demanded skills which we’ve had to cultivate across the company, and we’ve been sharing that knowledge with our customers. The bet we’re taking here is that the majority of the market will become even more tech savvy and data driven.”

While we’re marking the general availability of BQuant Enterprise, we’ve barely passed Act I. Our engineers continue to build new BQuant features and capabilities, such as a managed-service offering and apps to perform analytics on additional asset classes. Our BQuant teams are also growing and hiring. Visit here to learn more about their open roles.

Special thanks to Tom Barnwell, Mario Cadete, Chakri Cherukuri, Sylvain Corlay, Bruno Dupire, Shawn Edwards, Kevin Fleming, Jason Grout, Javier Mitra Valdes, Cheryl Quah, Andrey Rybka, and Arun Verma for their contributions to BQuant and to this article.