Elementarya startup building a data platform based on the popular Dagster director, announced today that it has raised a $33 million Series B round led by Georgian. This round also saw participation from new investors 8VC and Human Capital, as well as existing investors Sequoia, Index, Amplify, Hanover and Slow. The new round brings in the company’s total funding $48.8 million.
As so often, Dagster founder Nick Schrock also founded Elementl after many years at Facebook, where he also co-founded GraphQL. Schrock currently serves as the company’s CTO and chairman, while his former Facebook colleague Pete Hunt is now the company’s CEO. As Hunt told me, he had invested in Elementl as part of the 2017 seed round – mostly as a bet on Schrock. Hunt admitted that at the time he didn’t really understand Dagster’s value proposition, but as he worked through more data issues at Facebook and later at Smyte, the anti-abuse service he co-founded and later sold to Twitter, the need for a better data orchestration soon became clear to him.
Image Credits: Elementary
“I realized there are big complex data pipelines making very serious decisions – not just deleting social media posts, but deciding who gets a mortgage, all that sort of thing. Once you reach a certain size, every company is a data company and every company has a data platform,” said Hunt. This also means that managing their data pipelines is one of the biggest challenges for many companies.
Apache airstream remains one of the most popular tools for building these pipelines (and there are plenty of startups betting on it), but Schrock was looking for a more modern system optimized for the world of cloud, DevOps, and containers. But the team also rethought data pipelines from a higher perspective. “The way people have built data pipelines in the past is that they think in terms of tasks. So step A to step B – and then do step C. Within those steps they can do anything and you don’t really know – they can write to a database in a way you don’t expect and you have no way of controlling that or be observable in that step,” Hunt explained.
Image Credits: Elementary
Elementl has rethought this with what it calls a data asset (which can be a table in a data warehouse or a file in a data lake) at its core. So instead of thinking about tasks like the core abstraction, Elementl (and Dagster) focus on the data assets. “Putting this idea of an asset at the core of our system gives us a ledger of every data item in the organization and every state transition it has gone through, along with all associated metadata. That’s a mental model that developers love,” says Hunt.
Given that it competes with well-tested tools like Apache Airflow, Dagster should also work well for large organizations – and it should also be a legitimate open source project. Like most open source startups, the company is bringing business features like single sign-on, role-based access, and team support on top of the open source project as it builds out its commercial offerings. And since Airflow is so popular, the team has recently joined launched a tool which allows current Airflow users to run data pipelines written for Airflow on Dagster.
Over the course of the past year, the number of active projects using Dagster has tripled, the company says, as has the overall open source community around it. Currently, companies like DoorDash, Flexport, and Aritzia use Dagster in production.
“Dagster was built from the ground up to deliver a transformative developer experience while supporting the most demanding use cases in data engineering. Our unique abstractions and asset-first approach really resonate with data practitioners, and we see this happening
in our key growth metrics,” said Schrock.
The company plans to use most of the new funding to build out its go-to-market organization.
“Our R&D team adopted Dagster for data orchestration more than a year ago after evaluating the solutions in space. We are impressed with how Dagster accelerated our engineering team’s productivity and ability to efficiently ship production-quality data pipelines.” said Emily Walsh, lead investor at Georgian.
