Portland, Oregon-based startup thatDot, which focuses on streaming event processing, today announced the launch of Quine, a new MIT-licensed open source project for data engineers that combines event streaming with graph data to create what the company calls a “streaming graph.”
That sounds complicated, in part because it is and also because it’s a relatively new concept. Based on years of research that was supported by DARPA, the idea behind Quine is to take high-volume data streams and build them into stateful graphs. Quine, which was — as you surely already guessed — named after logician Willard Van Orman Quine, can then query this graph using what the team calls a “standing query.” These are essentially real-time computations on the incoming data that Quine can then, in turn, stream out to other applications.
“We’ve developed the streaming graph to really target the kind of the problem in the industry right now — the rock and hard place that we all sit between,” Quine’s creator and thatDot CEO and co-founder Ryan Wright told me. “On one side, there’s huge volumes of data. For the last 10 years, big data has just become de rigueur, it’s a normal ordinary thing now and only getting bigger. But the other side of that is how do you interpret all that data?”
More often than not these days, that data is in motion, and for many workloads, latencies matter. Wright argues that existing solutions like the open source Apache Kafka event streaming platform in combination with Apache Fink for analyzing that streaming data force enterprises to devote dozens of engineers to build their data platform and pipelines to analyze all of this incoming data. As for other modern approaches, there are tools like Neo4j and TigerGraph, which have popularized graph databases among developers, but Wright argues that they all approach this problem from the perspective of a database.
“In that mode of thinking, you’re stuck with all the traditional old problems that you’d encounter about the technical details and difficulties of making that fast and making that easy and making that scalable. So what usually happens in the industry is, if you’ve got lots of data coming in, you can’t really consider graph solutions because they’re too slow,” Wright said. Most current solutions, he argues, would be able to maybe handle a few thousand events per second, but the kind of customers thatDot is talking to would need a solution that can handle 250,000 events per second and Wright is confident that Quine can handle that — and a lot more.
“Using Quine, I replaced pages of complex custom logic and SQL queries with simple queries for the stream computed rollup value that updates at each underlying event change,” said Matt Splett, a principle engineer at Tripwire.
Those users, Wright and his co-founder and COO Rob Malnati noted, span the gamut from security companies to observability companies, log processing companies, fintech companies, advertising companies and fraud detection companies. Like other open source companies, thatDot’s mission is to support high-volume enterprise Quine users, but the company is also using Quine as a platform to build new solutions on top of.
The company raised just over $2 million in seed funding in 2020, led by the Oregon Venture Fund, and expects to raise a Series A round later this year.
Comment