I’m not going to restrict myself to answers here. Sometimes I just have good questions. Here’s one. I was talking to a CTO yesterday who wanted my reaction to having engineering spend 30% of their time “paying down technical debt”. My reaction was immediate, “Terrible idea!”
But why?
I don’t know the answer but I suspect it has something to do with time.
The value of tidying peaks just before the behavior change the tidying facilitates. If I Tidy First today, then the tidier system is shiny and clean when I change behavior tomorrow. Net Present Value tells us that spending later and earning sooner are both ways to earn more.
The cost of tidying is minimized around, before or after, the behavior change facilitated by the tidying. Just before we know just what needs tidying, so we avoid overdoing it. Just after the code is fresh in our minds and we know exactly what the design would have had to have been for the behavior change to go more smoothly.
Maybe when we take time out of the goal we are prone to tidying the wrong amount at the wrong time. There’s also a psychological/social aspect to it that’s escaping me at the moment.
I can imagine thinking this is a great idea or a terrible idea, depending on the situation. Let's pretend for the moment we are all friends here and agree in principle on the value of Shu-Ha-Ri.
If we consciously choose to tidy as a form of practice and habit-building, then I love it. We can tidy in a purely speculative way, because we are practising tidying and built the habit of wanting to tidy. We are choosing to tidy when it is safe and calm in order to prepare ourselves better to tidy under pressure when it is difficult.
This kind of speculative tidying has uneven and maybe unpredictable value. This provides more benefit when (1) we're only rehearsing and don't need to perform or (2) the code base is so bad that tidying any of it is likely to have benefit or (3) we don't know how to assess the hotspots of the code base and we use this speculative tidying as a way to map out the territory. However we approach this speculative tidying, we'd best limit our investment of time and energy, because otherwise some programmers fall in love with tidying and forget to ship features while other programmers find it tedious and give up just before it gets good.
This seems to make the advice or choice to "set aside 30% of our time for tidying" helpful mostly for focusing on building up production capacity. Once a group has enough of that, it seems wise to switch to a much more focused, tactical, just-in-time approach, where the design improvements both feel more compelling (we're just about to change this and now that's much clearer!) and more responsible (we're focusing on the part that needs to change next).
But... how do we know that we have enough production capacity to swing from speculative to purposeful tidying? I don't know, but I can guess. Try both. Typical human boredom will probably lead the average programmer to switch to purposeful tidying, but when volatility in the amount of tidying feels "too high", then it's worth budgeting time for some speculative tidying, either because they need to work more efficiently (practise! practise!) or they don't yet have enough of a habit of tidying as they go. This means some combination of (1) setting aside time to tidy as a separate stream of work and (2) more cleaning up before moving on in their daily work. I encourage timeboxing both.
As the programmers gain more experience, have more-deeply-ingrained habits, and more finely-honed judgment, they merely tidy as much as they need where they need it and when they need it. They find their balance easily. In this context, setting aside a fixed budget for tidying probably doesn't make much sense, unless they need it to recover from having consciously decided to rush temporary in order to meet a short-term milestone.
I also think it depends on other factors. What is the goal of the change? How important or critical is the change? How big is the impact of the change, both technical and functional wise? In what area will the change occur? Who will make the change and who will do the tidying? How much time will the tidying take versus how much time will making the change take? Is it worth it?
Some parts of the code will never or hardly ever change. Some parts are prone to changes every month or every year.
Let's say tidying your home is needed. But for what occasion? If it's just daily/weekly routine it's not that much. But what if guests will come over? A birthday party or some other joyful event to celebrate. You want the living room, the kitchen and the bathrooms neat and tidy.
But in case you have guests sleeping over, you also need to tidy their guest space. Depending on the duration of the stay it is also needed to accommodate other things.
Another situation might occur where you want to sell the house. In that case quite an overhaul has to take place. Not to only tidy it, but to make it presentable, marketable. And not just the living room, but everything, the whole shebang (#!). Until the house is sold, it can take ages before the strict tidying can loosen up a bit again.