Check out our new case study with Taktile and dltHub
Check out our new case study with Taktile and dltHub
Check out our new case study with Taktile and dltHub
Hello, world!
Nov 25, 2024
In a few hours we are going on stage at our design partner’s dltHub big reveal party (check this link to see if there are any spots left) for the Portable Data Lake. It’s going to be a sort of an unveiling for ourselves too (and if you read this post to the end, you will understand the angry chicken picture). We will be together with another great design partner of ours, Taktile, who will share their story of a 1-person data team becoming 10x more productive by using dltHub’s powerful dlt+ technology and running it locally and in production using Tower.
Tower was founded by Brad Heller and me because we sensed that the data engineers of today were looking for a much simpler development experience that took advantage of the powerful hardware of their Macs and Dell XPSes. These data engineers were Python-first and preferred to work with a new set of Python frameworks such as dltHub, Polars, Datafusion and others. But this came at a price.
A local dev experience introduced a bunch of friction when multiple team members had to sync credentials to data sources and environment settings. Another problem was that once a data app was developed locally, there was no easy way for deploying it into a production environment that mimicked their local environment. People were building custom data platforms out of proverbial sticks and duct tape. And so the idea for Tower was born. Tower would run Python data apps reliably in production, and seamlessly integrate with the development flow.
We were also acutely aware that an increasing number of data platform teams in enterprises and digital natives were looking to unify their data storage using open table formats like Iceberg, and enable their users to employ the query engines that were best for their workloads. We worked at AWS Database Services when Netflix was incubating Iceberg and was pushing their IaaS vendors to open up to the idea of vendor-independent data storage. Large customers can indeed move their vendors to do things that benefit the entire industry.
Brad and I met at Snowflake where we launched performance-boosting features in the Snowflake control plane. Our claim to fame is releasing a feature that reduced the execution time of customer queries and saved customers hundreds of millions of dollars annually (hello, consumption-based pricing), temporarily wiping out about $10B in Snowflake’s market cap after an infamous quarterly earnings call where this feature was “blamed” for slowing down revenue growth. While the revenue slowdown caused some pretty intense moments internally, Snowflake as a platform became much more competitive in the High-Concurrency Low-Latency field and, judging by the latest market moves, is doing quite well.
Brad is a startup founder with multiple exits (incl. to Puppet), who’s been building production platforms all his life. He's also been working with data since the very beginning. His very first business trip, straight out of university, was to Kampala, Uganda, to deploy a data collection and aggregation system he built with Microsoft. The system conducted surveys over SMS and was used by midwives and the Ministry of Health to save babies' lives in remote parts of the country. Unbeknownst to Brad, at about the same point in my own career, I was working with the Ontario Ministry of Health on a system called Healthy Babies Healthy Children that did pretty much the same thing but in the more snowy conditions of Canada.
In addition to bonding with Brad over the baby health thing, we share a passion for building tools for data people. I've been building database and data processing products first at AWS (relational databases), then Google (unified batch and streaming processing with Cloud Dataflow), Snowflake (HCLL and Search) and Databricks (Spark-based compute governed by Unity Catalog). Between the two of us, we literally talked to thousands of customers who treat data processing as a critical part of their businesses. As former founders, it was easy for us to agree on a fundamental tenet for how we approach building the Tower product: we are looking for a strong customer signal before starting the work on a feature, and won't invest our time building something unless we sense a strong customer need.
We situated Tower originally in Berlin, Germany, because it is full of engineering talent and is a great city to live as a young professional (the music scene! the food!) or someone with a family (playgrounds everywhere!). However, we think of ourselves as a global startup that serves the needs of customers everywhere. Expect to see us at the Small Data events in SF, at Re:Invent, at the Moscone Center…
For this post we picked a photo with the sign of a local Berlin eatery because (1) Tower was officially founded at the notary public office above the eatery and (2) “Angry Chicken, So So Angry”, in our opinion, should be the motto of every startup embarking on their journey. We promise an ever increasing number of updates that will make the life of the new generation of data engineers easy and fun. This here chicken will be so so angry.
Wish us luck at tomorrow’s demo. Talk soon!
In a few hours we are going on stage at our design partner’s dltHub big reveal party (check this link to see if there are any spots left) for the Portable Data Lake. It’s going to be a sort of an unveiling for ourselves too (and if you read this post to the end, you will understand the angry chicken picture). We will be together with another great design partner of ours, Taktile, who will share their story of a 1-person data team becoming 10x more productive by using dltHub’s powerful dlt+ technology and running it locally and in production using Tower.
Tower was founded by Brad Heller and me because we sensed that the data engineers of today were looking for a much simpler development experience that took advantage of the powerful hardware of their Macs and Dell XPSes. These data engineers were Python-first and preferred to work with a new set of Python frameworks such as dltHub, Polars, Datafusion and others. But this came at a price.
A local dev experience introduced a bunch of friction when multiple team members had to sync credentials to data sources and environment settings. Another problem was that once a data app was developed locally, there was no easy way for deploying it into a production environment that mimicked their local environment. People were building custom data platforms out of proverbial sticks and duct tape. And so the idea for Tower was born. Tower would run Python data apps reliably in production, and seamlessly integrate with the development flow.
We were also acutely aware that an increasing number of data platform teams in enterprises and digital natives were looking to unify their data storage using open table formats like Iceberg, and enable their users to employ the query engines that were best for their workloads. We worked at AWS Database Services when Netflix was incubating Iceberg and was pushing their IaaS vendors to open up to the idea of vendor-independent data storage. Large customers can indeed move their vendors to do things that benefit the entire industry.
Brad and I met at Snowflake where we launched performance-boosting features in the Snowflake control plane. Our claim to fame is releasing a feature that reduced the execution time of customer queries and saved customers hundreds of millions of dollars annually (hello, consumption-based pricing), temporarily wiping out about $10B in Snowflake’s market cap after an infamous quarterly earnings call where this feature was “blamed” for slowing down revenue growth. While the revenue slowdown caused some pretty intense moments internally, Snowflake as a platform became much more competitive in the High-Concurrency Low-Latency field and, judging by the latest market moves, is doing quite well.
Brad is a startup founder with multiple exits (incl. to Puppet), who’s been building production platforms all his life. He's also been working with data since the very beginning. His very first business trip, straight out of university, was to Kampala, Uganda, to deploy a data collection and aggregation system he built with Microsoft. The system conducted surveys over SMS and was used by midwives and the Ministry of Health to save babies' lives in remote parts of the country. Unbeknownst to Brad, at about the same point in my own career, I was working with the Ontario Ministry of Health on a system called Healthy Babies Healthy Children that did pretty much the same thing but in the more snowy conditions of Canada.
In addition to bonding with Brad over the baby health thing, we share a passion for building tools for data people. I've been building database and data processing products first at AWS (relational databases), then Google (unified batch and streaming processing with Cloud Dataflow), Snowflake (HCLL and Search) and Databricks (Spark-based compute governed by Unity Catalog). Between the two of us, we literally talked to thousands of customers who treat data processing as a critical part of their businesses. As former founders, it was easy for us to agree on a fundamental tenet for how we approach building the Tower product: we are looking for a strong customer signal before starting the work on a feature, and won't invest our time building something unless we sense a strong customer need.
We situated Tower originally in Berlin, Germany, because it is full of engineering talent and is a great city to live as a young professional (the music scene! the food!) or someone with a family (playgrounds everywhere!). However, we think of ourselves as a global startup that serves the needs of customers everywhere. Expect to see us at the Small Data events in SF, at Re:Invent, at the Moscone Center…
For this post we picked a photo with the sign of a local Berlin eatery because (1) Tower was officially founded at the notary public office above the eatery and (2) “Angry Chicken, So So Angry”, in our opinion, should be the motto of every startup embarking on their journey. We promise an ever increasing number of updates that will make the life of the new generation of data engineers easy and fun. This here chicken will be so so angry.
Wish us luck at tomorrow’s demo. Talk soon!
