Mahendra Ramsinghani
More posts from Mahendra Ramsinghani
The Linux Foundation recently announced the launch of the Reactive Foundation. Its founding members are Alibaba, Lightbend, Pivotal and Netifi.* So what exactly is this Reactive Kool-Aid, and why are all these companies guzzling it down so fast?
If you buy the premise that developers live in a cloud-native microservices world, then you also understand that most applications are distributed and elastic. The compute is spread across clusters, as is all the data. It could be a few users, or a spike of thousands. Systems need to be architected to handle these spikes. Yet the dark secret of microservices is complexity — the management of resources, costs, performance and latency remain a challenge.
If we break down any application into hundreds of moving parts (such as containers and microservices), then we better have an elegant way to manage those moving parts. These services need to talk to each other, exchange data and ensure that overall performance is reliable, at all times. Easier said than done.
The “big unsolved problem of the cloud”
According to Daniel Berg, Distinguished Engineer at IBM Cloud, “The network is the unsolved problem of the cloud…. We need the network to be a first-class citizen of a cloud system.” Why does the network remain a problem? Is it because we fall back on our old ways, when we need to rethink the new? We have designed the car with the big clunky wheels of a horse buggy. Conceptually, it sounds fine — but it can be a pretty rough ride.
In the layered cake of network protocols, we have the middle layer of transportation (Transmission Control Protocol / Internet Protocol or TCP/IP), and right at the top, we have the application layer. We use a protocol called hypertext transfer protocol (or HTTP) to make sure the web applications can talk to each other. TCP was born in 1974 and is called a “chatty protocol” — it has to go back and forth many times just to do some basic stuff. A TCP joke circulating around proves this point.
HTTP came in 15 years later, in 1989, and was used to serve documents in a client-server era. This was when we all had desktops being cooled with whirring fans. We would use a Netscape browser to launch a web page (hypertext) and the web-server would say, “Wait a sec – let me fetch that for you.”
Three decades later, we are trying to make do with HTTP, when the compute layer has exploded. Does HTTP work in the world of millions of interactions with machine-to-machine communications? Our mobile, IoT and edge devices are not quite requesting pages and walls of text. And there is no client-server as much as peer-to-peer exchange. But the network layer is stuck with us and we are trying to make sure these microservices can stay put using some archaic methodologies. “As much as 89% of all microservices architecture is based on HTTP, says Stéphane Maldini, principal software engineer at Pivotal. Pivotal is one of the founding members of the Reactive Foundation. In the process, we are creating a big trade-off in efficiency. We are still using two cans and a piece of string to communicate, when we should use the next iPhone.
HTTP is unsuitable for microservices
If we use HTTP in the micro-services world, we have fundamental challenges. For one, there is no flow control — “which means that data flows from a fire hose,” says Robert Roeser, co-founder of Netifi. Because the data can be dumped at a rapid pace, and multiple threads are opened up, we end up building control features to ensure the application does not crash.
Stuff like circuit breakers, retry logic, thundering herd (where a large number of processes wake up, but only one wins, often leading to freezing up) needs to be managed effectively. In HTTP, everything is a request / response, but if we look at a simple notification for an app, we don’t need to keep polling all the time. The request is like a grumpy kid sitting in the backseat whimpering, “Are we there yet?,” when the journey has just begun.
Such inefficient mechanisms cause a huge waste of compute resources when we use the wrong protocol. IBM documented the inefficiency of microservices and concluded that the performance of the microservices is ~ 79% (s)lower than the monolithic model. “We identified that Node.js and Java EE runtime libraries for handling HTTP communication consumed significantly more CPU cycles in the microservice model than in the monolithic one,” conclude the researchers.
Goodbye HTTP, hullo Reactive
The Reactive Foundation sits under the Linux Foundation and aims to accelerate the next generation of networked technologies. It espouses the merits of Reactive Programming Frameworks and builds the community. Ryland Degnan, chair of the Reactive Foundation and co-founder of Netifi, lived the HTTPain while he was a member of the Netflix edge platform.
Ryland understands scale, latency and user experience better than most people. At Netflix, the platform would have billions of requests from over several hundred million members. He says, “We live in a multi-dimensional universe where user experience matters. Developers have to deal with three axes of (a) deployments (b) frameworks and (c) protocols. Spotty connections are unacceptable. Why can’t we pick the stream up from where you left off? If we do that alone, we reduce 90% of our infrastructure.”
Indeed, Facebook has adopted RSocket to reduce the dropped connections over mobile network hops and reduced its edge infrastructure significantly. Steve Gury, a software engineer at Facebook speaking at SpringOne Platform said, “The future is R-Socket.”
Reactive programming is a paradigm shift at the architectural level. It’s about speed and performance. One of the major strengths of Reactive is asynchronous I/O, which allows reduction of edge infrastructure by orders of magnitude.
Andy Shi, developer advocate at AliCloud (a unit of Alibaba), is one of the founding members of the Reactive Foundation. He says, “Alibaba has thousands of developers as we are one of the world’s biggest e-commerce platforms. As we adopt microservices and see that compute is utilized only around 10%, throwing more infrastructure at the service mesh is not the answer. Pods are talking to each other using REST API which is not the way to go.”
REST APIs require multiple endpoints and round trips to get the data. Another founding member of the Reactive Foundation, Viktor Klang, deputy CTO at Lightbend, has been evangelizing Reactive for well over a decade, and feels like the time has finally come. “Our systems need to produce results in the required time frame. Imagine if you could compute an answer to a grand question — like the meaning of life — but if the answer is delivered after you die, the system has failed,” he says.
Comparing service meshes and use cases
While Istio is the 18-wheeler truck best suited for lift and shift, RSocket is the Ferrari — speed and elegance. Experts foresee a world where the two may coexist. Yet there are applications, such as IoT use cases, where RSocket has a clear edge (pun intended). Istio offers load balancing, service discovery, logging and traffic management but with heavy overhead.
In studies, Netifi was able to process 16X more requests and delivered four times higher throughput in comparison while maintaining three times better latency — 372% faster throughput with 300% less latency. “Netifi has the potential to be like a Cisco — the router of the microservices,” says Creighton Hicks, investor at Dell Technology Capital.
Istio was launched by Google, IBM and Lyft, so it is a strong incumbent and with serious brand cachet. But when the likes of Alibaba and Facebook start to showcase the RSocket ROI, the fun has just begun. During a recent presentation in London, the Reactive mafia was in full swing. Ondrej Lehecka, a software engineer at Facebook, and Andy Shi talked about how RSocket is addressing real-world architectural challenges. Shi said, “RSocket is designed to shine in the era of microservice and IoT devices. Projects built on top of RSocket protocol and Reactive streams in general will disrupt the landscape of microservices architecture. The Reactive Foundation is the hub of these exciting projects.”
*Mahendra Ramsinghani is an investor in Netifi.
Comment