Gardening with Tracer Bullets
< td bgcolor="#000000">
Pragmatic Programmers Andy Hunt and Dave Thomas talk with Bill Venners about a gardening metaphor for software development, the reasons coding is not mechanical, and the importance of getting feedback during development by firing tracer bullets. This article features some highlights from a sequence of conversations between Andy, Dave, and Bill that originally appeared on Artima.com.
Andy Hunt and Dave Thomas are the Pragmatic Programmers, recognized internationally as experts in the development of high-quality software. Their best-selling book of software best practices, The Pragmatic Programmer: From Journeyman to Master (Addison-Wesley, 1999), is filled with practical advice on a wide range of software development issues. They also authoredProgramming Ruby: A Pragmatic Programmer's Guide(Addison-Wesley, 2000), and helped to write the now famousAgile Manifesto.
Software Development as Gardening
Bill Venners: In your book, The Pragmatic Programmer, you say, "Rather than construction, programming is more like gardening." I really like your gardening metaphor for software development. Can you elaborate on it?
Andy Hunt: There is a persistent notion in a lot of literature that software development should be like engineering. First, an architect draws up some great plans. Then you get a flood of warm bodies to come in and fill the chairs, bang out all the code, and you're done. A lot of people still feel that way; I saw an interview in the last six months of a big outsourcing house in India where this was how they felt. They paint a picture of constructing software like buildings. The high talent architects do the design. The coders do the constructing. The tenants move in, and everyone lives happily ever after. We don't think that's very realistic. It doesn't work that way with software.
We paint a different picture. Instead of that very neat and orderly procession, which doesn't happen even in the real world with buildings, software is much more like gardening. You doplan. You plan that you're going to make a plot this big. You're going to prepare the soil. You bring in a landscape person who says to put the big plants in the back and short ones in the front. You've got a great plan, a whole design.
But when you plant the bulbs and the seeds, what happens? The garden doesn't quite come up the way you drew the picture. This plant gets a lot bigger than you thought it would. You've got to prune it. You've got to split it. You've got to move it around the garden. This big plant in the back died. You've got to dig it up and throw it into the compost pile. These colors ended up not looking like they did on the package. They don't look good next to each other. You've got to transplant this one over to the other side of the garden.
Dave Thomas: Also, with a garden, there's a constant assumption of maintenance. Everybody says, I want a low-maintenance garden, but the reality is a garden is something that you're always interacting with to improve or even just keep the same. Although I know there's building maintenance, you typically don't change the shape of a building. It just sits there. We want people to view software as being far more organic, far more malleable, and something that you have to be prepared to interact with to improve all the time.
Coding Is Not Mechanical
BV: You say in your book, "Conventional wisdom says that once the project is in the coding phase, the work is mostly mechanical, transcribing the design into executable statements. We think this attitude is the single biggest reason that many programs are ugly, inefficient, poorly structured, unmaintainable, just plain wrong. Coding is not mechanical." You seem to feel strongly about that.
DT: The ultimate objective of writing software is to produce code that generates some business value. And that value is produced by writing code. It's only at the time that you write the code that you know, yes, this is actually working. Before that time, it's all theory. You can do your best to validate your designs and check your architectures, but really, it's all theory until it actually runs. In the running of the code, you get the feedback that you need to know you're going in the right direction.
What happens if you don't do that? What happens if back at the design level you say, "I'm an omnipotent designer. I can produce things. I know they'll be right. These are just coding details"? You are eliminating an entire level of feedback. And I think it's actually the most valuable feedback you can get, because it's the feedback that says, "This is what the program does."
Quite often, you'll be programming some requirement that should be simple, but turns out to be really complicated. Why is this? Often, it is because the requirement has been poorly stated or understood. You say, "Wait a minute, I don't like how this is turning out." You go back to the client and say, "You asked for this, but it is having all these ripple effects in the code. Why is this?" And the client will say, "Oh, I didn't really mean it that way. I meant it this way." The code is a mirror, a model of the reality you're actually implementing. If the code has problems, it could well be because the reality, as expressed to you, has problems. Simply seeing code as mechanical loses all the benefit of that feedback.
AH: As an anecdote, on one project Dave and I were implementing legal requirements in code. These were actual laws that had been passed. We got about three quarters of the way through it and realized that the law as stated was wrong. It was inconsistent. If you read through the law one way, it said something had to happen. But in the next paragraph, it contradicted itself. Nobody had caught it up to that point, because nobody had tried to do a detailed implementation enacting this legal requirement. This mere mechanical act of coding actually catches bugs in the real world. If you dismiss it as just a mechanical translation, what are you going to do? You find genuine requirements problems while coding. You've got to be able to react and adapt to them.
Stratification of Developer Jobs
DT: The other dimension to this whole issue — and this is something I feel very strongly about — is the stratification of job titles. The idea that you have business analysts, architects, designers, coders, and then people considered the scum, the testers, is a really major cause of problems in the industry, for a number of reasons. First of all, you have the problem that no one feels responsible, because it is always somebody else that's handing the stuff to you.
AH: Or somebody that you're going to pawn it off onto next.
DT: Right. You also have a phenomenal communication overhead. Every single thing has to pass up and down the chain. Everything has to be a memo: the business analyst to the architect, the architect to the designer, designer to coder, coder to tester, tester to coder.
AH: And you lose signal through each interface, and introduce noise.
DT: Right. You get this children's party game of Telephone, where a requirement goes in on one end. Every time it gets transferred from one level to the next, it gets subtly distorted. You start out wanting an order entry system, and you end up with a payroll system by the time it gets down to the programmers.
BV: What is the alternative to stratification of job roles?
DT: I don't think you can get rid of the layers. I think politically that would be a mistake. The reality is that many people feel comfortable doing things at a certain level. What you can do, however, is stop making the levels an absolute. A designer throws a design over the partition to a set of coders, who scramble for them like they are pieces of bread and they are starving, and they start coding them up. The problem is this entrenched idea that designers and coders are two separate camps. Designers and coders are really just ends of the spectrum, and everybody has elements of both. It's similar with quality assurance (QA). Nobody isjust a tester.
What we should be doing is creating an environment in which people get to use all their skills. Maybe as time goes on, people move through the skill set. So today, you're 80% coder, 20% designer. On the next project, you're 70% coder and 30% designer. You're moving up a little bit each time, rather than suddenly discovering a letter on your desk that says, "Today you are a designer."
AH: If the designer also has to implement what he designs, he is, as they say, getting to eat his own dog food. The next time he does a design, he's going to have had that feedback experience. He'll be in a much better position to say, "That didn't quite work the way I thought last time. I'm going to do this better. I'm going to do this differently." Feedback paths contribute to continual learning and continual improvement.
BV: Continual learning is part of the notion of software craftsmanship?
AH: Exactly. First you've got the designer doing some coding, so she'll do a better design next time. Also, you really need to plan how you are going to take some of your better coders and make them designers. As Dave says, will you give them "the letter" one day? Sprinkle some magic fairy dust and poof, they're now a designer? That doesn't sound like a good idea. Instead you should break them in gradually. Maybe they work with that designer, do a little bit of this design today, a little bit of that design tomorrow. Some day they can take on a greater role.
BV: In your book, The Pragmatic Programmer, you suggest that programmers fire "tracer bullets." What are tracer bullets? And what am I trading off by using tracer bullets versus "specifying the system to death," as you put it in the book?
DT: The idea of tracer bullets comes, obviously, from gunnery artillery. In the heavy artillery days, you would take your gun position, your target position, the wind, temperature, elevation, and other factors, and feed that into a firing table. You would get a solution that said to aim your gun at this angle and elevation, and fire. And you'd fire your gun and hope that your shell landed somewhere close to your target.
An alternative to that approach is to use tracer bullets. If your target is moving, or if you don't know all the factors, you use tracer bullets — little phosphorous rounds intermixed with real rounds in your gun. As you fire, you can actually see the tracer bullets. And where they are landing is where the actual bullets are landing. If you're not quite on target — because you can see if you're not on target — you can adjust your position.
AH: Dynamically, in real time, under real conditions.
DT: The software analog to firing heavy artillery by calculating everything up front is saying, "I'm going to specify everything up front, feed that to the coders, and hope what comes out the other end is close to my target." Instead, the tracer bullet analogy says, "Let's try and produce something really early on that we can actually give to the user to see how close we will be to the target. As time goes on, we can adjust our aim slightly by seeing where we are in relation to our user's target." You're looking at small iterations, skeleton code that is non-functional, but enough of an application to show people how it's going to hang together.
Basically, it all comes down to feedback. The more quickly you can get feedback, the less change you need to get back on target.
AH: Having said that, there are times when either approach is appropriate. You might think it sounds silly taking the big guns and firing computers approach. Well, if you're shelling a city, which isn't moving, that's a pretty good way to go. If you're writing software for the space shuttle, or any kind of environment where you really know all the requirements up front and they're not going to change, that's probably the cheapest way to do it. And there are some environments where that is the case. Not many, but that actually does happen. If you're shooting at something that's not a city, something that's moving, then you want to get that rapid feedback.
DT: Almost by definition, your target is moving. The sheer act of delivering the first release is going to make the user realize, "Oh, that's not quite what I wanted."
AH: Or even worse, "Oh, that's exactly what I wanted. But now having seen that, I've changed my mind. I've learned. I'd now like to do this instead, or this, in addition." Just by introducing the software, you've changed the rules of the game. Now the user can see more possibilities that they weren't aware of before. The user will say, "Oh, if you can do that, what I'd really like is if you could do this." And there is no way to predict that up front.
Starting with a Skeleton Application
BV: You say in your book, "In tracer code development, developers tackle use cases one by one." What is a use case, and why do you recommend that we do tracer bullets in use case chunks?
AH: Central to tracer-bullet development is the idea of a skeleton application, in which one thin line of execution goes end to end. In a skeleton application, you have some bit of functionality -- even if it's just the equivalent of "Hello, world!" -- that goes all the way from the UI through business logic, through whatever else is in the middle, and all the way to a database. It may do nothing more than put a checkbox value into the database as a Boolean. Whatever the skeleton application does, it does end to end, skeletally thin.
From the skeleton application forward, you might start adding one feature at a time, where each feature implements one use case. You implement the feature, go all the way through, and see if it hangs together. As you steadily grow your application, one feature at a time — this is where tracer bullets come in — you may realize that you're off by a little bit. Well, you still don't have that much code. It's still easier to adjust it and fix your aim. As you grow the application fatter and fatter over time, it will be increasingly harder and harder to make gross changes. But you'll also be getting smarter as the project progresses, because you are firing tracer bullets and getting feedback. So when the requirements are more volatile, the code is easiest to change. Later, the code does become harder to change, but by then you know more and less change is required.
This interview includes excerpts from a ten-part interview with Andy Hunt and Dave Thomas originally published on Artima.com:
- "Don't Live with Broken Windows: A Conversation with Andy Hunt and Dave Thomas, Part I"
- "Orthogonality and the DRY Principle: A Conversation with Andy Hunt and Dave Thomas, Part II"
- "Good Enough Software: A Conversation with Andy Hunt and Dave Thomas, Part III"
- "Abstraction and Detail: A Conversation with Andy Hunt and Dave Thomas, Part IV"
- "Building Adaptable Systems: A Conversation with Andy Hunt and Dave Thomas, Part V"
- "Programming Close to the Domain: A Conversation with Andy Hunt and Dave Thomas, Part VI"
- "Programming Is Gardening, Not Engineering: A Conversation with Andy Hunt and Dave Thomas, Part VII"
- "Tracer Bullets and Prototypes: A Conversation with Andy Hunt and Dave Thomas, Part VIII"
- "Programming Defensively: A Conversation with Andy Hunt and Dave Thomas, Part IX"
- "Plain Text and XML: A Conversation with Andy Hunt and Dave Thomas, Part X"
Andy Hunt and Dave Thomas are authors of The Pragmatic Programmer, which is available on Amazon.com:
Andy Hunt and Dave Thomas are also authors of Pragmatic Unit Testing and Pragmatic Version Control, which are available at their very own Pragmatic Store.
Ward's Wiki, the first WikiWikiWeb, created by Ward Cunningham