LinkedIn open sources Flashback, a tool for mocking internet traffic

LinkedIn, the social network for professionals that is now owned by Microsoft, is making one of the tools that it has developed in-house for its own work open for use by others. The company today announced that it would be open sourcing Flashback, a tool for mocking internet traffic for developer tests, under a BSD two-clause license.

In a blog post, LinkedIn said that it had been using Flashback to test things like reliability, scalability, and speed on new code before deploying it more widely. You can see an example of how it works here.

Flashback was based on Betamax — not the old video standard, but a another tool developed to mock actions of web services and REST APIs in developer tests by “intercepting HTTP connections initiated by a web application, and then later replaying them.” (‘Betamax’ because it was inspired by another tool, the VCR library for Ruby.)

The key difference, write creators and LinkedIn engineers Shangshang FengYabin Kang, and Dan Vinegrad in a blog post, is that Flashback can work in an isolated environment, whereas Betamax, like other proxies, requires internet connectivity to work.

“Flashback is designed to mock HTTP and HTTPS resources, like web services and REST APIs, for testing purposes. It records HTTP/HTTPS requests and plays back a previously recorded HTTP transaction—which we call a ‘scene’—so that no external connection to the internet is required in order to complete testing,” they write.

The reason for building a tool that didn’t require a connection to the internet was because of problems they encountered when using Betamax. These included the fact that LinkedIn’s test environment does not have internet access for security reasons; and the code had use cases that required authentication protocols like OAuth and OpenId, which required complex HTTP-based interactions that didn’t work with Betamax replays.

This is not the first time that LinkedIn has outsourced some of the code that it originally developed for proprietary purposes. For example, in 2015, it also outsourced FeatureFu, a toolkit for building machine learning models.

The reason that LinkedIn open sourced FeatureFu was because it was no longer seen as a “business differentiator” for the company. That may have been the case here, too, but so was the fact that when LinkedIn presented the tool publicly, they saw requests from the developer community to use it for their own projects.

While LinkedIn is open sourcing Flashback, it’s also going to continue to work on it to expand its functionality — which could also point to what LinkedIn itself may be working on down the road for its 465 million users.

“We’d like to see if we can support non-HTTP protocols, such as FTP or JDBC, in the future,” they write, “and maybe even give users the flexibility to inject their own customized protocol using the MITM proxy framework. We will continue improving the Flashback setup API to make it easier to support non-Java languages.”