In response to last Friday’s email, A monolithic success story, Rick Yust responded with the following on LinkedIn:

I love new test environments for every deployment; for every PR is even better, and again even better for the manual testing in isolation.

That said, there’s also room for long running environments since often times there are stale data or configurations which don’t become visible in fresh environments. Large data sets (eg: companies with thousands of users) might not be adequately tested when a fresh test environment creates sample companies with a handful of users. Similarly, missing or changed objects or properties or fields of database or json payloads can cause responses to behave unexpectedly before recordsets are fully migrated. These kinds of details can easily be missed when you’re working with freshly built environments, but can quickly become apparent when you’re working with large sets of existing or historical data.

There are development approaches I’m sure can mitigate those risks, but I will always believe there’s a good reason to have at least 1 long running, non-production but production-like environment to run manual or automated tests against prior to production deployment.

Rick brings up some great points!

One approach that can sometimes help with the issue of needing realistic data in a test environment is to seed each new environment with data from production, or with a curated data set. When seeding from production, this can often raise privacy concerns, if you’re dealing with PII, credit card numbers, or other sensitive data. Some teams have set up auto-scrubbing and/or randomization to help solve this. YMMV.

I’m also fully in favor of a long-running staging or test environment for other reasons. One is dog-fooding for products where it’s not possible or reasonable for employees to use the production system. It’s also great to use a staging server for demos and exploratory testing or configuration.

In summary, there’s no single one-size-fits-all solution to all testing needs. Ephemeral review environments are great, and I believe should be used more. But that doesn’t mean they solve all problems. Most teams will also want a dedicated, long-lived staging or test environment in addition!