I got a great question from a reader of my daily email list, which I want to take a stab at today. Shared with permission.

I’m working through your archives now on trunk-based development (TBD) and thinking through it from the lens of data engineering.

What’s the relationship been successful TBD and a microservice architecture? Is TBD less feasible in a monolith environment?

I’m struggling conceptually to figure out TBD in data engineering and analytics environment where maintaining state is fundamental and the systems are necessarily tightly coupled.

Most of my data engineering experience has been with feature branches, but I’m highly interested to explore TBD and apply it to my work in data.

Regards, Sawyer Nyquist

There’s quite a lot riding in this question, and there’s no way I can address all possible nuances in a short post like this. But to most directly answer the question, is trunk-based development less feasible in a monolith environment? No, I don’t think so.

However, not all monoliths are created equal! And some of the patterns that make trunk-based development difficult may be more common on monorepos or monolithic applications.

Sawyer eludes to some of these patterns in the rest of his question: maintaining state and tightly-coupled systems.

The benefits of trunk-based development, and the other practices that build on it, such as continuous integration and continuous delivery, definitely can be realized with monoliths, monorepos, and in data engineering. But you’ll need to find solutions to these obstacles.

And to start, I’d challenge the assertion that these systems are “necessarily tightly coupled.”

There are different types, or dimensions, of coupling. And while two (or more) components of a system may need to exist together for the system to function, and I suspect this is what Sawyer means, this doesn’t need to mean the components are “tightly coupled” in the sense that matters for things like trunk-based development and continuous integration.

What we care about in this context, is whether the systems must be changed together. We care whether or not they are temporally coupled with regard to changes. And to enable TBD and CI, we need to break that temporal coupling such that it can be safe to change one part of the system without changing all other parts of the system that depend on it.

This is in no way a challenge unique to data engineering.

In fact, it’s a challenge on even the most mundane of old-school monolith web applications, which is often overlooked and/or ignored entirely. When you update your web application, there’s a chance (often a near certainty) that some user has an old version of a page loaded in their browser at the time you upgrade the backend service. If that old page makes assumptions about the backend that are no longer valid (such as linking to a page you’ve deleted), you’ve fallen vistim to temporal coupling.

The solution, then, is to make sure that every change you make to any depended-upon component, is backward-compatible with the current expectations of those components that depend upon it. You maintain this backward compatible state long enough for all dependant components to be updated to the new set of expectations. This might be a few minutes (as with most web apps), or days, weeks, months, or even years (as when supporting old versions of the Windows operating system).

But this is the “one, not-so-simple trick” that lets you break that temporal coupling, and enable TBD or CI, whether building a web app, microservices, a monolith, or anything else.

Of course there are likely other challenges. The key for whatever challenge you face is to ask: What would need to change to allow us to do trunk-based development?

Often the answer is surprising. And, at least in my experience, less scary than I initially imagined.