Bazel: Aspirations and Realities
Avoiding traps when developing in a new startup.
As a fervent advocate of Bazel, I’ve experienced its exceptional capabilities in established environments, when done properly it makes you fly as a developer. Yet, my journey of integrating Bazel into two greenfield startups has been a reality check. Despite my affection for it, the substantial initial setup time repeatedly proved to be a daunting barrier in the fast-paced startup landscape.
A Beacon of Hermetic Builds
Bazel’s allure lies in its promise of hermetic builds. The concept is straightforward and compelling: given a compatible OS and a C compiler, Bazel should seamlessly execute a complete system build. This is not just a matter of convenience; it’s a cornerstone of reliability, ensuring consistent build executions and shielding the process from external disruptions, such as outages in external repositories like PyPI or GitHub.
Bazel’s true forte emerges in monorepos, where its ability to map dependencies shines, simplifying the navigation of complex architectures. Moreover, Bazel’s deterministic nature allows for efficient caching of build artifacts, significantly accelerating both builds and tests.
The First Attempt: A Lesson in Prioritization
My initial tryout with Bazel began in 2019, during the nascent stage of my first startup. Fresh from Spotify, where a significant shift towards Bazel was underway — as detailed in the blog post “Switching Build Systems, Seamlessly” — I was keen to implement what I perceived as modern best practices. Looking back this has been the wrong approach, what I should have been doing was building an MVP as quickly as possible, and later on after achieving some sort stability in growth begin to fortify my position with a solid build system. There is also an element of nerd-sniping that comes from working on build systems, its very technically satisfying and the solution to your problem is always just a programming step away (as opposed to something more esoteric like product market fit).
The challenge was amplified by the state of Bazel support for Python and JavaScript — the primary languages of my project. These languages, not being compiled like C or Java, were somewhat sidelined within the Bazel community, resulting in a strenuous and ultimately abandoned effort to integrate Bazel effectively. This eventually lead to me giving up and using a monorepo with requirements.txt files referencing other local project and Lerna for the javascript side.
The Second Endeavor: Elevated Aspirations
In 2023, armed with extensive Bazel experience from my tenure at Google, I embarked on my second startup adventure. This time, my confidence in handling Bazel from the ground up was bolstered, with a project primarily based in Python, supplemented by C++ and R.
My vision extended beyond Bazel. I admired Google’s development ecosystem, particularly its use of Remote Build Executors and Remote Caching for consistent and swift builds, and the flexibility of developing through Cider on lightweight devices. Copybara’s role in managing third-party dependencies was another aspect I planned to incorporate.
The startup, a hedge fund, demanded a build system like Bazel for its definitive repeatability — a non-negotiable in the high-stakes financial world. The explicit control over dependencies Bazel provided was crucial to avoid the pitfalls of indirect versioning.
Embracing Bazel’s philosophy of building from first principles, I embarked on building Terraform with Bazel — a formidable endeavor given its extensive web of dependencies. Despite initial optimism, the process became a Sisyphean task, with each converted dependency only marginally edging us closer to an elusive efficiency.
The challenges extended to converting Python projects, particularly when faced with intricate Cmake build rules. The effort to translate these into Bazel was both time-consuming and complex, ultimately hitting a wall of diminishing returns.
Reflections and Moving Forward
In retrospect, the quest for a perfect Bazel implementation of my project was in vain. The lack of widespread Bazel adoption in open-source projects and the absence of a more extensive Bazel Central Repository for dependencies were significant hurdles. The realization dawned that successfully implementing Bazel in a startup environment would likely require a dedicated team and considerable time — a luxury often unaffordable in the startup world.
Despite this, my belief in Bazel’s potential remains unshaken, particularly for specific domains like my latest venture. For now, I’ve turned to a combination of poetry in a monorepo setting and docker compose, to provide a reliable interim solution for interconnected component testing. The Bazel journey isn’t over; it’s merely paused, awaiting a time when the scales of project stability and available resources tip in its favor.