Hearken to me, my fellow developers. We live in complex and confusing times. Once we sought to make things ever simpler. Then we discovered that simple isn’t necessarily scalable, or efficient, or resilient, and turned to favoring composition over simplicity, deconstructing our systems into multiple independent services. And now–well–now it sometimes seems that we have hit upon the worst of all worlds.
That would explain the eruption of enthusiasm for what has quickly become a whole genre of darkly funny technical blog posts that made the rounds this month, (fondly) mocking modern software development and the paralyzing ever-growing profusion of tools, systems, and frameworks, a plethora crossed with a Lernaean hydra. I have been known to call the resulting selection anxiety Developaralysis.
CircleCI‘s founder Paul Biggar led off the parade of dark technical comedy with a post that’s already a classic of its kind, proclaiming “It’s The Future!” Yes, indeed it is. Be afraid. At the risk of comic spoilers, let me cite a sample–
–actually, you should click through and read the whole thing yourself first, to get its full effect–
–last chance to do so–
–OK, fine, here’s the highlight cite:
So I just need to split my simple CRUD app into 12 microservices, each with their own APIs which call each others’ APIs but handle failure resiliently, put them into Docker containers, launch a fleet of 8 machines which are Docker hosts running CoreOS, “orchestrate” them using a small Kubernetes cluster running etcd, figure out the “open questions” of networking and storage, and then I continuously deliver multiple redundant copies of each microservice to my fleet. Is that it?
To which a ruefully amused audience of techies around the world groaned, and laughed, because the alternative was to weep and rend their hair and wear sackcloth and ashes, which as I understand it remains something a fashion faux pas even in liberal Northern California.
Others took up the sardonic torch. Inspired by Biggar, Boopathi Rajaa wrote an excellent follow-up piece on Medium, lampooning modern JavaScript frameworks and front-end development:
Okay. So I’ve got dozens of tools and libraries for building, bundling, transpiling, whatever. Now what?
I was saying. Flux — It lets you architect your applications well. With Observables and Immutable data, it gets nicer and interesting.
Observable ? I need immutable data ?
I expect many more posts in this genre to emerge. There is so much to send up.
But at the same time, like all good comedy, they’re getting at something serious here, which Biggar thoughtfully examines in a follow-up post:
It’s very easy to see why people might think the container ecosystem is bullshit, in exactly the way I satirized … microservices which are new and shiny but seem like a fantastically stupid idea considering how hard it is to keep a single service running in the first place … Most haters are not really reacting to Docker’s solutions to important and complex problems. Mostly, this is because those problems are ones you might not have noticed if you haven’t spent time scaling big systems.
I would dispute some of his conclusions. (For instance, I think it is possible to combine arbitrary scaling with something close to Heroku’s simplicity. In fact I’d argue that Google App Engine, for all its flaws and infelicities, does just that.) And I’m personally inclined to–where possible–wait for the new container ecosystem to mature a little into something more PaaS-like before diving deep into it. But he’s quite right that you can’t paper over, or abstract away, the real problems of building resilient distributed systems.
The larger problem is that the various subfields of software engineering have become so broad and complex that just keeping up with them, much less mastering them, is pretty close to a full-time job. So we use tools that abstract and simplify the complexity for us…but that’s always a tradeoff. There’s nothing wrong with making that tradeoff, but you need to be aware it exists. It has ever been thus, since the first compiler was written.