In order to run complex algorithms on data at any scale you normally need a lot of engineering resources and management. Like, a LOT. Over 50% of the total amount of money spent on processing that oft-remarked thing called Big Data is spent on engineering and developer operations. It’s a big problem to solve, and basically a sitting duck for a startup to attack. The problem is, it’s an incredibly complex problem to solve and so far all we can do is throw the sheer brute force of bodies and resources at the problems.
However, a new company in London is making a very, very bold claim. It says it’s solved this problem. The promise is to get away from using software, to run algorithms at massive scale with zero engineering, over the command line.
This may sound far-fetched, but it’s clearly a claim that has convinced investors, since Hadean has now closed a seed round of $2.6m. The round was led by White Cloud Capital, with Entrepreneur First participating. Hadean was founded by Rashid Mansoor (CEO) and Alec Mocatta (Chief Architect).
Mansoor says: “Hadean is an operating system that empowers individual developers to iterate rapidly at cloud-scale, unleashing unprecedented productivity gains.”
Hadean says a single developer can deploy and run code at any scale using their existing tool chain (Rust, C, Linux etc.) – and without the need for any of the ops or tuning traditionally associated with Big Data.
If they, and their investors are right, it might have just levelled the playing field for startups to compete against the biggest companies, allowing them to side-step the massive engineering costs required to get an algorithm to work on big data at scale.
Here’s their claim: They’ve essentially, re-engineered the entire computing stack to put supercomputer levels of processing power at the disposal of anyone with a laptop and internet access.
They claim to have removed the need for Spark, Hadoop et al and any cluster management software or solutions.
With Hadean, a single developer logs into the cloud-based platform and can instantly run complex algorithms on data at any scale without the need for any engineering resource or management. Yes, it’s an extremely bold claim, and one which the wider tech world is going to have wait to test.
Because right now, the platform is in beta, with a public release due “later this year”. That said, the company is currently working with a small number of customers in finance (a “big bank”) and a genomics research team (“sequencing genomes”).