Here’s an update on my journey deploying Apache Superset and using Docker. It felt good to write about what I’ve learned at work, and it’s been a while: I don’t think I posted here that my job title is now Senior Data Engineer! But people who don’t work on computer infrastructure might not find this one interesting.
In 2023, we deployed Apache Superset at the City of Ann Arbor as our Business Intelligence (BI) / data visualization platform, choosing it over Microsoft Power BI or Metabase. That decision has been a resounding success. Superset is a rock-solid product that keeps getting better … and* we’ve saved over $150k and counting in license costs vs. proprietary software.
I’ve re-read the 2015 talk “Choose Boring Technology” a couple of times while working in my current job (that link goes to a slideshow version of the talk, turned into a website – that’s the format I’ve experienced it in). The author talks about having only three innovation tokens to spend on new tech at a given time. The rest has to be boring. Then when you’ve mastered the new tech, you get a token back to spend on something else.
Deploying Superset took all my tokens: Docker, DevOps, Linux sysadmin. I said, we will only deploy official Docker images released by the Superset project. No way are we in the business of creating our own, this is complicated enough.
I learned a ton in the intervening years. I’m still learning a ton. It’s great! As I’ve gotten those tokens back by becoming competent at those technologies, I’ve been able to do more with our Superset deployment.
First that looked like building our own Superset Docker image, tweaking the environment but not touching the code. The project forced our hand on this because starting in 4.1.0, it no longer included basic drivers needed to use Superset out of the box, most notably the one to connect to the PostgreSQL backend database. I’m still not entirely convinced this was the right choice for the project but I saw the other side’s argument that everyone really ought to be building their own image.