J.B. Rainsberger is a long-time XP practitioner, who believes in helping developers simplify their work lives.

In this first part of a two-part interview, J. B. joins me to talk about evolutionary design, what it is, why it's useful, and the barriers that keep many people from experiencing its benefits.

Resources
Test-Driven Development by Example by Kent Beck
Programmer Anarchy talk by Fred George
Working Effectively with Legacy Code by Michael Feathers

Guest
J.B. Rainsberger
Personal Site
The World's Best Intro to TDD
Blog

Transcript

Jonathan Hall: Our focus today is some of the psychology related to refactoring.

Operator: Ladies and gentlemen, the Tiny DevOps guy.

Jonathan: Hello, everybody, welcome to another episode of the Tiny DevOps Podcast. I'm your host, Jonathan Hall and as always, we talk about DevOps for small companies and small teams. Today, I have a special guest, J. B. Rainsberger, who I've been following for several years off and on. Welcome, JB, thanks for coming on. Why don't you give us just a brief introduction to what you do and why you might know something about the topic we're discussing today.

J. B. Rainsberger: Yes, thanks very much, Jonathan. [unintelligible 00:00:40] the chance to chat with you and I guess the short version is, I'm an old XP guy. That's extreme programming and I became interested in not so much DevOps specifically, I'm not as big a part of that community as maybe I could otherwise be but a lot of what we were talking about in the early days of extreme programming, fairly logically flow into what DevOps has become and at least that idea of the programmers having more input and even control over their environment.

I happened to become interested in this stuff because I wasn't a very good programmer, I felt like it was really hard for me to write 10 lines of code without making a mistake. Things like test-first programming became interesting to me. Over the years, my general approach to software really hasn't changed that much from the Extreme Programming days, those fundamentals have really served me well over the last 20 years.

One thing that's changed a lot is the overall tone message framing of what those techniques like test-first programming like evolutionary design, like continuous releases, how that changes the way that programmers think about and experience the professional software development and so what's really emerged over the last 5 to 10 years, for me, has been to focus on helping programmers work with a lot less stress, help them feel-- I want to help them feel better about their craft, not necessarily because I want them to become masters, not necessarily because their performance and their business results are that important to me, although they are for any clients listening.

I genuinely believe that great results come from programmers who have slack time and energy, to care about their craft without being asked to do it, who feel like they have enough time to do good work to protect their capacity to deliver features. A lot of what lies at the center of that for a lot of programmers is stress and so a lot of the techniques that I teach and write about, although they do produce good results, at their core, it's really about helping programmers work with less stress so that they can get out of their own way and do the kinds of things that will help them be better employers and better employees and suppliers to their employers and clients.

Jonathan: That's great. A lot of people in software development these days work with Scrum. To many people, Scrum is the face of Agile Software and that's all they know and I've experienced, I imagine you have too, I'd love to hear if it's not, that many people don't realize that there are a whole other set of practices that help them on a technical level, they think that they have backlogs and they have story points.

They have product owners and Scrum Masters and they don't realize that there's a whole other school of resources and thought about doing the technical part of agile, better. I'm really excited to talk to you today about how we can deep into that and how people especially those who've never heard of this idea that you mean there's more to agile than just Scrum. What can we do? How can we take something away from this and improve?

J. B.: Yes, I know that our focus today is probably going to be a little bit more on refactoring and some of the psychology related to refactoring but it is important to me for people to understand that what we call is complicated but what we have been trying to call Agile software development for the last nearly 25 years now consists of a lot more than Scrum, there are various-- We think of them as schools. Just like in martial arts, you have various schools just like in painting, you have various schools where people may be focused on certain kinds of techniques over others.

What we all have in common is this lightweightness what really Agile meant to be from the beginning was exemplified by what's the least we can do and still deliver great results. Ultimately, that's where it came from it was attempts to revolt against the heavyweight ceremonial trends of the '80s and '90s and so get back to a little bit more of what it must have been like for people who build software professionally, in the early days of the profession, when maybe the one programmer had to know how to do everything, and therefore, they had to be good at everything and they had to care about everything.

They were involved in every aspect, when we stratified you testers do this, you coders do this, you designers do this, you architects do this, your business analysts do that. The basic theory of constraints, thinking will tell us that all those handoffs are going to create delays and those delays are going to create stress and suffering, we don't want to do that and that's one of the reasons why I want to reclaim the concept of lightweight software development. In part, because lightweight does a better job of capturing what it is that we all have in common Scrum, XP, FTD, DSDM, all the various school's Crystal.

Lightweight is a one really important thing that we have in common, humane is another but and we'll get into that in a moment and whereas maybe the word Agile has become misconstrued and has become the new heavyweight way of approaching things or in some camps, as you mentioned, it has become mostly equated with sticky notes and retrospectives and product owners sticking their nose in where it doesn't belong and all these other stuff.

If I get to wave my magic won that the next decade would involve a back to the basic principles approach where we try to just go back to that classic question, what is the least we can do, and still deliver great results, and then look at our context and decide which of the various techniques from extreme programming, from Scrum from Feature Driven Development from DSDM, from lean Theory of Constraints, any of these areas, Kanban Method, whatever it is, and bring those to bear.

As I said, it so happens for me that, because I'm more interested in helping people at an individual level, managing their energy, in particular, the stress level is where I've landed on that and so one of the reasons that I want things, I like lightweight because it also makes-- We feel lightweight, when we work with less stress, more focus, more concentration, worrying about one thing at a time and I think that-- There's one big thing I've noticed over the last 10 years is that a lot of programmers are trying very hard to figure out how to be good at their craft, and they feel overwhelmed with all the things that need to learn now.

It's much harder to be a programmer now than it was 20 years ago That's added a lot of stress and then asking them to do things like Master evolutionary design so they can protect their capacity to deliver features over the long term, while also fighting with the people who are worried that they're not delivering features quickly enough. That's enough stress anybody back to bed and that's one of the things that I'd really like to help them with.

Jonathan: This is a topic that I get asked about a lot, DevOps, we decided-- Someone decided that it was a good idea to take everything that you're talking about in Dev and add Ops under the same so-called job title. We say that DevOps isn't really a job title, but it feels like it sometimes and I get asked this all the time, "Jonathan, how do I learn enough to do DevOps? How do I learn everything I need to know about solid principles and in TDD, and also TerraForm and Kubernetes, and all these million other things?" [chuckles]

Yes, we need to approach this in a lightweight way. It seems like some of this comes easy for some people and not for others. What's the difference? Why do some people just run with it and they seem to just maybe they look like from the outside, they look like a superhero? They're doing everything, and others of us, we struggle and we have all this complexity. What's the difference?

J. B.: Ah, that's a huge question. Maybe I'll try to answer a small part of it because the question that you asked, right there, I don't know. I don't know what it is that makes it easy for some and not for others but I have noticed one specific thing that seems to happen. That is an indicator of the ones who stick with it and see the benefits and realize those benefits, and in particular, thinking about something like the evolutionary design, and the ones who get stuck in this perpetual advanced beginner state.

I'm using Trifexis model of skills acquisition, so when I say advanced beginner, that's actually stage 2, I'm not making judgments about people. I know nobody likes to be called a beginner, but an advanced beginner is somebody who mostly has left the stage of needing to be directed on every aspect of what they're doing. They're starting to figure out how to put some of the pieces they've learned together in ways which are novel for them where nobody taught them you have to do this, followed by this, followed by this.

If you think of someone who's interested in cooking or baking this is the person who's starting to make substitutions in the recipes that they've been following, and they don't always get it right. They don't necessarily know the theory behind how to balance flavors, but they at least understand that if I don't have honey I can probably use maple syrup and get away with it.

Different hot peppers will probably be okay, and I'm going to have to play around a little bit to figure out which hot peppers are too hot for this kind of dish, then I can learn little things like, well, if I use peppers that are too hot, then a little bit of cinnamon or sweetness can help balance it. Starting to learn a little bit of the theory, but still mostly when things get tough, when you're not sure what to do you still need to look outside for help.

You're past the stage of having to follow the recipe exactly, but you're not yet at the stage where you can answer a lot of your own questions. That's what I mean by the advanced beginner stage. What I've noticed is that a lot of programmers get stuck in the advanced beginner stage when it comes to evolutionary design and specifically when it comes to refactoring. A person knows that they're stuck in the advanced beginner stage when a couple of things are going on for them.

One of them is that they have this vague idea that they should be able to make room for a feature they're trying to add, or make it easier to fix a defect. By making some relatively small changes to the structure of the code they're working on, but it's so difficult, I shouldn't say difficult. They still have to put so much conscious effort and attention into the steps of refactoring into remembering what keystrokes in their IDE were, extract method or rename or move, or if I'm working in a plain text editor what are the steps that I need to do in order to be able to safely move this function from that module.

That module move this method from that class to that class. When it comes time to actually execute the steps they still require some conscious effort. It's not easy. They can figure it out. They might need to remind themselves how to do it, but they haven't yet attained that point where the fingers seem to do it even before the brain entirely engages. That's one thing.

Another thing is that as a result of that they have this feeling that, "Oh, now I know how I should have designed this part of the system. Let me think about how to refactor to get from here to there." Then they sit and think about it for a while. They have this vague feeling of dread like I can't quite see how to get from here to there in a bunch of discreet, reversible, safe, easy-to-understand steps. I know how to throw this part of the code away and write it again, and hope that I get it done soon enough that nobody will be delayed by me doing that, but then that's not really refactoring. That's not really the benefit of evolutionary design.

Part of the benefit of evolutionary design isn't just that we can refactor the code, but is this confident feeling that I can keep the design simple for today's problems, and next week when we encounter a feature we didn't have in mind, or when we have to fix a problem we didn't realize was coming. We can just adapt the design to meet the needs of the future without anticipating the needs of the future. We can every week pretend that we made the right design decisions all along.

That's really part of what refactoring is meant to enable. It's not meant to just be a mechanical way of changing design. Imagine how powerful it would be if you could pretend that you always made exactly the right design decisions at every moment. Yet, at the same time, didn't have to anticipate the future. That's what evolutionary design is supposed to be about. I think programmers because they don't develop this facility with the craft--

I was going to say the craft, but really more I mean the techniques involved. They get stuck in this feeling of, "Oh, I should be able to refactor here. Oh, it will take me too long. Oh, I'm feeling product owner X or boss Y breathing down my neck. I know how to do it the fast way even though it's not the right way it'll probably be good enough. I'll try again next time."

Then, of course, as we know, next time rarely comes. If I never feel like I know how to take advantage of evolutionary design to fix this problem right now, or to make room for that feature coming up in the next day. If I never practice it I never get good enough at it to feel confident at it which means that I'll never actually do it. This is how we end up in this advanced beginner state forever.

I might be willing to refactor as an exercise an hour or two every so often if it's a task that nobody's breathing down my neck for, but I'll never feel comfortable using it in an industrial-strength situation under real-time pressure when it could actually be valuable when it could really have an impact. Let alone convincing the people around me to join me and doing it which is a whole other problem.

Jonathan: [chuckles] Indeed. What do you say to the naysayers about refactoring, or more important to the point the evolutionary design? It would save us so much time if we would just do the big design up front there's that school of thought too. In my experience, most of those people have tried evolutionary design in some way and found it to be a disaster. How do you address these people?

J. B.: A couple of things. One, I think I heard this from Ron Jeffries for the first time 20 years ago which was if I knew which mistakes I was going to make in advance I would've won the lottery by now. That's one thing. One of the reasons why we can't just build the whole thing up front or design the whole thing front I should say is that we have this really strange habit programmers due of not noticing that the picture in our mind isn't quite right until we actually start writing code that's when we [unintelligible 00:17:17]. We've heard this in a million different ways no plan survives contact with the enemy.

I think that the primary difference between the people who favor evolutionary design and the people who favor design upfront is that the people who favor evolutionary design have accepted the idea that you probably can't understand it all in that much detail upfront, that there is some amount of learning that happens when you build it. Some of that is simple stuff that you just didn't see coming. In principle, if you had thought about it for two weeks longer, you might have figured all that stuff out.

At the same time, it's very hard to understand everything at the highest scope and at the smallest level of detail and be comprehensive about the whole thing. That's asking a lot of any human. Do we really want the only competent software designers in the world to be the people who have this rare cognitive capacity to hold all of those details in their head at once? I don't think we need to put that limitation on us.

The people who insist on designing more upfront are really betting the whole enterprise on some key people's ability to have that, or some key people's cognitive capacity. I just don't think we need to do that. One of the things that we've learned about evolutionary design having practiced it for a long time is that once you get past that state of advanced beginner nest, Evolutionary design is expert architect fabrication system. It helps people really deeply understand how to design systems well.

They're designing systems better every day while they're learning. They deeply understand what makes modular design work. They build tremendous judgment which allows them to make difficult trade-off decisions in key moments. More important than anything else, everyone in the project community can learn how to do it. I genuinely believe that everybody can learn to be competent at evolutionary design and that it's actually not that hard to become really quite good at it.

Here's where I could insert the I did it so can you joke I'm not. I'm going to say the opposite. I was one of those people who was in the right place at the right time, and I do have that cognitive capacity. I'm a pretty smart guy. I have a lot of raw genetic talent if you want when it comes to just being a pattern recognition machine and being able to hold a bunch of things in my head and I can't do design up front very well. I'm going to say the opposite, not that because I can do it other people can do it, but more like I'm really bad at design upfronts. They're pretty laughably bad at it.

If I can't do it, then I'm not so sure that other people should try that hard and I certainly don't want them to feel like failures if they can't. That's really for the naysayers, I would say, I understand your pain I know why it happens. The thing that want to talk about today, the thing that we're here to discuss this sort of, how do you get past that advanced beginner stage of evolutionary design is exactly the reason why you shouldn't trust it, because if you don't know how to help people get over that hump, if they're going to be stuck as advanced, like a whole bunch of advanced beginners, doing refactoring on a project is a disaster.

We need some people who get past that advanced beginner stage and that's part, that's really one of the key messages that I've been trying to deliver over the past couple of years is what's going on? How do we get past it? That I can say to the naysayers, once one person gets over that hump and then a second person gets over that hump, it's profit after that. That's what makes this really so critical and so, unfortunately, we have this happen in software development of throwing a bunch of beginners at things and saying, Ah, they'll figure it out." Evolutional design is one of those things for which that simply doesn't work.

This isn't the same as learning about how some technology works. This isn't the same about as learning, how some framework or some library works where you just learn it, document it, and hope that other people understand the words you wrote. There's a real craft of this. It's the difference between learning chess and playing it at a high level. I don't want to overstate how hard it is, but I do want to be clear that if we just let people stay advanced beginners and expect good results, we're going to have a bad time. The good news is I have some ideas about how we can help people get past that advanced beginner stage and in that case, I am going to say, if I can do it, so can you.

Jonathan: Before we go into those tips, could you recap, if you're a listener, how does a listener identify that they are an advanced beginner with evolutionary design and refactoring?

J. B.: I think the short version is that if you feel like you don't need to follow step-by-step guides for every refactoring, if you feel like there are some refactoring that you understand how to do, but occasionally need to be reminded, need to remember exactly which keystrokes they are, need to remember exactly which sequence the steps are, if you try a refactoring that's of say, not one that's routine, but if you try a refactoring, that's not routine for you yet, and you make mistakes and you have to undo and try again.

If it seems like you never progress farther than that, that you're focusing so much on how to refactor safely, that you never feel like you can think clearly about, am I going in a good direction? Is this really going to improve the design? That's a signal of being stuck in that advanced beginner stage where you are so you're still so worried about the mechanics that it feels like you're just moving code around and nothing's actually getting better.

Jonathan: We've identified that we're now an advanced beginner, what can we do? How do we start to advance to the next stage?

J. B.: I'll start a little bit more about why it's important to be aware of this and what bad things can happen and that's the idea that-- When you're stuck in this advanced beginner stage, what tends to happen is you sit down to maybe add a feature or fix a defect, or you see some problem in the design that you want to improve. You have some spare time, you want to try to be a good product community citizen, and you look at what it is that you would need to do.

You don't have a clear plan of exactly how it's going to work. You have some basic idea of, I want to remove duplication, I want to improve names. I feel comfortable that I know how to maybe move code from place to another, but I don't know what the key abstraction that's missing is. I don't have a really big idea about where I want the design to go. I just know that this design makes that change difficult, and I want to move in that direction.

Then you sit down to do it and you think about, okay, roll up your sleeves, start doing a couple of small refactorings, extract a method, move a method from this class to that class, maybe rename something. You start to feel like I'm not really sure that this is actually getting any better, but now I have a clearer picture in my mind of what I wish it looked like. Then maybe you take some paper and index card or something, you draw a diagram of what it is that you actually think the design should look like.

You look at the code, you look at the diagram of where you want to go and it feels like I don't know how I'm going to get from here to there. I can pick some decent sized part of the code, throw it away and try again or I can try building this new thing and hoping that I can figure out how to make the wires fit into the existing system later but then that starts to feel like, "Oh, that's going to take me a long time. I wasn't really planning on spending three days doing this. I wanted to make some improvements in a couple of hours."

When this happens, one of the thoughts that comes to your mind is I can't afford to refactor this code right now. If you keep saying that to yourself often enough, then eventually, that becomes something more like I'm never going to be able to refactor this code. If you say that you're self often, if that starts to become something like every time we make a design decision in this code base, we have to be really careful because we are going to be trapped by those decisions for the rest of our lives, or at least for as long as I'm on this project.

We're never going to get out from under the design choices that somebody made in the past or that we made today. We become terrified that we're going to essentially create constraints for ourselves that we're never going get away from. If you say that often enough, then it starts to become something more like, I just don't think I'm ever going to be able to do good work around here. Every time we make a design decision where it becomes a prison it's, I don't think this is the right place for me.

Jonathan: It sounds like you're talking about the other side of the coin, when you hear lean practitioners say things like defer decision or implement something that you can change later, that's only valuable if you know how to change it later. Right?

J. B.: Absolutely true. Part of the point of refactoring is not just to make the design better, but part of the point of evolutionary design, like I said earlier, is to allow us to pretend that we made the perfect decision at every stage, because if we make decisions less expensive to change, that doesn't just allow us to recover from changing conditions or to allow us to recover from actual mistakes, which do sometimes happen. We just have the wrong idea of what we need and we do it incorrectly and we need to fix it.

Think about how much freer you feel when you have confidence that you can change your mind, that changing your mind is not going to cost an exorbitant amount. That it's not a failure, it doesn't bring judgment with it and it's something that you can just change your mind almost whenever you need to. There's a dark side to that too, but the bright side of it really is that if I feel comfortable changing my mind, then I don't have to agonize over this decision right now. I can afford to make a decision, which seems reasonable for right now, knowing that I can change it later.

That sounds like in a nightmare until you actually experience it and you realize that the amount of rework that you were terrified was going to happen actually tends not to happen. Yes, some rework does happen. Yes, there are some decisions that we make in the heat of the moment that end up not surviving even to the next day, but the 85% of the time when we just easily guide the design to evolve, gives us a lot of spare energy to deal with the 15% of the time when we actually mess up.

What that does is that wears down our resistance. That makes it much easier for us to actually benefit from evolutionary design. The word helplessness that I was describing earlier, the flip side of that is what happens when you really actually start to get good at this after a while. It's not just about being able to move code around, but it's about you start to feel like I don't have to agonize over every decision. The number of decisions that I need to agonize over goes way down the number of decisions, which are costly to change goes way down.

We just have a lot more freedom to do what we need to do in the moment, knowing that we can make it fit the design better, literally, minutes later or weeks later and it costs almost the same and that's incredibly freeing. I think it's stress-reducing on balance that it really helps people feel like, you know that feeling of I have to be careful about this decision because we'll fix it later, but later never comes? As we improve at evolutionary design, at refactoring in particular, later actually comes.

Part of the reason later number never comes is that we feel like it's very expensive to change our decisions. If we have hundreds and hundreds of examples of being able to change our decisions, less expensively, correctly, accurately, safely when we need to. Actually, we can defer commitment and feel confident that it's the right decision. It's not something we're getting away with anymore. It becomes actually a strategy instead of a concession. As you say, that's exactly what all the lean folks are telling us about real options thinking.

If we don't learn those techniques, we'll never unlock that power. To get back to your question of how do we actually do this? I have good news and bad news. The good news is that by practicing evolutionary design, we are constantly bathing in example after example after example of good design, better design, bad design, worst design. Every time we try to refactor we're adding a little bit more to our rich mental model of what makes design good. Now the bad news is if you only rely on the job practice as it were and I'm going to be very clear here. If you only rely on live practice under real pressure on a code base for which they pay you money, then your success will come down to luck.

This goes back to a question you asked earlier about why does it seem easy for some and not for others? One of the things that seems to happen is that if you're in the right place at the right time and you have enough slack and you believe in the value of evolutionary design, then you'll create opportunities to practice it in the way that I'm going to describe.

If you don't then you won't and if you don't practice deliberately, you're probably not going to get very far except by accident. Very few people can just read some books about evolutionary design and then do it well. If you have no slack to learn to practice in a deliberate way, then that is almost certainly going to trap you in that state of advanced beginner.

Jonathan: What is the relationship, in your mind, between a refactoring evolutionary design and TDD?

J. B.: In the old days we talked about test-first programming where the only rule was you don't write production code until you have a failing test. Maybe another way to say this, you don't add behavior to the system without a failing test. That's it, test-first programming, the way it was written in the late '90s that's really all it was. That's the way I teach today.

The difference between test-first programming and test-driven development is the driven part.

In fact, to the point where even in the early days after Kent Beck published his Test Driven Development: By Example book where he-- That signaled the division. Before that book, we talked about test-first programming. After that book, we talked about test-driven development. I think part of what he was emphasizing in that book and the way that I think of it since is that test-driven development is essentially the test-first programming rule plus refactoring.

If you just do test-first programming, when does design happen? Well, either design never happens and you get a big ball of mud that works but eventually, just becomes increasingly difficult to maintain or you do all that design upfront and then you use tests to help you type the code into the computer correctly and that allows you to have a well-designed system, which is less expensive to maintain and we have the confidence that the code does what we thought we asked it to do.

As a rough guide, you can think of test-driven development as test-first programming plus refactoring. What really the driven part of test-driven development, I believe comes from being able to start to use the tests. Not only as a way to tell you if the code behaves the way you wanted but to use the tests as a source of feedback about the design. When I teach this, what I ask people to do, after they get used to test-first programming, is to think about this extra, I don't want to call it a rule but a guideline like the no code without a failing test. That's a rule.

If you're not doing that, you're not doing test-first programming. That's fine. We can still be friends and we can still go out on Friday but don't call it test-first programming if you're not going to write a test before you write a production code. For test-driven development, for me, what I ask people to do is to think about this. If you notice that a test is annoying in some way, slow, painful to understand, painful to write requires a lot of copy and paste has a bunch of excessive tests set up whatever it is.

When you look at a test and go there like, you don't like it for some reason, then I want you to suspect the production code design. I want you to say there's something about the production code design. I almost made the mistake of saying wrong. There's something about the production code design which might or might not be okay, that is causing that test to exhibit this thing that makes you go there.

Instead of saying that it's the test's fault, that there's something wrong with the test, let's look at the production code design and ask the question. If we change the production code design in a way that made that test smaller, more concise, easier to understand whatever it is, would that actually improve the design of the overall system? Would that also reduce our cost to maintain the system over time?

The test-driven development practitioner will assume the answer's yes, I will suspect that the production code is the problem until proven. That's really what distinguishes test-first programming and test-driven development. Then I can just say that test-driven development is just one particularly safe way of doing evolutionary design? In principle, you can do evolutionary design without tests.

I'm not sure I would invest my own money in somebody doing evolutionary design without tests. It seems risky. I did run for six months, a bunch of ensemble programming sessions through PubMob.com called Evolutionary Design Without Tests where we expressly explore this question. What happens if we try to do Evolutionary Design Without Tests, how does it feel? What problems do we encounter?

The short version seemed to be-- The conclusions we reached in six months were essentially that if you have experienced test-driven development practitioners, the habits that they've built when they've practiced TDD allow them to do evolutionary design without tests surprisingly well. However, the people who don't have those habits yet are perhaps going to struggle to learn how to change code in teeny tiny steps that are so small, that each step is obviously correct but they're what they have is a vocabulary problem.

They just haven't seen enough examples of, well, if I add zero there in a way that's plus 12 and minus 12, and that allows me to rearrange the coding this way and it gets easier. Or let me take that object and or let me take that value and put it in a list and then take it back out of the list right away. That will allow me to extract this function which will improve the design a little bit but those tricks, they just haven't seen enough of them yet. One of the ways to see them is to practice refactoring. One of the ways to safely practice refactoring is to have tests. One of the ways to ensure you have tests is to write them first.

That brings you back to you can refactor without writing tests but I wouldn't recommend it until you've spent a decade or so refactoring with tests. Then if I take them away from you, you'll probably survive. You'll probably actually do fairly well. Fred George and the program Anarchy Movement establish that, that you don't have to write tests your entire career but it's probably a good idea to spend a good five to 10 years trying it on a daily basis. The habits that you build are really going to be helpful.

Then if I have to drop you into code where you don't have the tests that you want or if I dropped you in a situation where writing those tests is unusually expensive or unusually difficult, you don't have to feel stuck. Again, that goes back to the advanced beginner issue? Being stuck in the advanced beginner stage, part of that comes from never quite getting to the point where you've built those habits that allow you to do that without tests, safely, accurately, effectively in a way that will make me feel comfortable paying for the project.

Jonathan: I hear you describing this and it really resonates with my own experience. I learned to do TDD because I was trying to refactor and I didn't feel safe about refactoring without tests. Of course, I'd been doing tests after programming for awhile. I write my code. I write some tests, it covers 50% to 80% of the use cases. Then I would always later discover oh there's that one corner case I didn't think of covering and so on and so forth. Then I think it was Michael Feathers's book on refactoring that really drove home the idea of TDD. I didn't run with it right away but I would at least write a test before I would extract it or maybe I would extract first but before I refactor the function or write a test.

That's really what led me to the TTD thing. That's why I asked you because I see refactoring and TDD is really going hand in hand, at least in my own experience and I was curious. You've convinced me that I am an advanced beginner and I need to get better at refactoring. You've explained why it's important. What's the next step on my journey to improvement?

J. B.: The big thing that's missing.

Operator: Tune in next week for the exciting conclusion. Our conversation took longer than expected but that's all right. It just means we need to turn this into two episodes. If you're not already, be sure to subscribe so you're notified when the new episode comes out. This episode is copyright 2021 by Jonathan Hall. All rights reserved. [unintelligible 00:40:37] . Theme music is performed by [unintelligible 00:40:41].

[00:40:44] [END OF AUDIO]