Test-Driven Development with “( Test && Commit ) || TestCodeOnly || Revert”

November 23, 2018

I learned through JB Rainsberger’s “The World’s Shortest Article on (test && commit) || revert” blog article that Kent Beck and Oddmund Strømme came up with a new workflow for developing software, inspired by the “Limbo strategy” for concurrent development with a distributed team, summarized quite succinctly as “test && commit || revert“.  A number of people have been talking about it.  Some have been trying it out to see how it works for them, rather than just engaging in armchair speculation.  And there have been some questions about how well the “test && commit || revert” process can work, given that it does not allow for the “Red” step of the traditional “Red, Green, Refactor” cycle of Test-Driven Development.

I find that adding one step, making it “( Test && Commit ) || TestCodeOnly || Revert“, enables using the well-known “Red, Green, Refactor” Test-Driven Development process, in a style that has been recommended by some for quite a few years.  Automating the process, as suggested by these expressions, would most likely improve our rigor and discipline in the TDD process.

So how does it work?

Kent’s “Test, Commit, Revert” expression, and mine, assume that software development is done in very short cycles, by writing a very small amount code, maybe just one line, “evaluating the expression,” and doing so again.

Each of the “words” in the expressions above are scripts or subroutines that do something and return a Boolean true/false result.

The “Test” action runs all the automated regression xUnit tests, and returns “true” if they all pass.  If one or more tests fail, it returns “false.”

The “Test && Commit” expression implies, due to “short circuit evaluation” of the logical “and” operation (“&&”) that the “Commit” script will only be run if “Test” returns true.

The “Commit” operation should save all the code changes you’ve made (since the last Commit), and return “true.”  This would write your changes to a shared source code repository, if you’re using one.

The next operation and script in Kent Beck’s version is “|| Revert”.  With short-circuit evaluation of this logical “or” operation, the “Revert” script will not be run unless the “Test” operation fails.

So Kent is suggesting that whenever any xUnit test fails, the system should automatically discard all the changes you’ve made to the code since the last time Test returned true.  That is, your code would revert to what it was the last time “Commit” was executed.

I happen to agree with Kent that this is fundamentally a good idea, in spite of the frustrations it may cause.  And this idea was proposed quite seriously by others quite a few years before this current conversation started.

It occurs to me that we can restore the traditional “Red, Green, Refactor” cycle, and improve the automation and rigor of recognized good Test-Driven Development process by adding only one more step to the expression — the TestCodeOnly step.

The “TestCodeOnly” step checks to see if you have changed only xUnit test code, or if your changes include the “Code Under Test.”  If you have only changed code in source directory holding the xUnit tests, then the “TestCodeOnly” step returns “true.”  But if you have changed any code at all in the “Code Under Test,” it returns “false.”  The “Code Under Test” is the “production code” that you intend to deliver in the working system.  The xUnit test code should be kept separate and not deployed into production.  This separation of “test” and “production” code is a widely used “best practice” that ensures a smaller, simpler, more well-focused production code release by avoiding the distraction and risk test-only code being deployed into production.

“( Test && Commit ) || TestCodeOnly || Revert”

So this is how this process would generally work:

You need to start with an environment where the xUnit tests pass.  For the first step in an empty environment, you’d write a single test that does nothing by pass.  Then evaluating the “( Test && Commit ) …” expression will get “true” for the “Test” running step, and “Commit” your code changes.

Write a very simple test, in typical TDD style, and evaluate the “( Test && Commit ) || TestCodeOnly || Revert” expression.  Because we have not written code to make the test pass, we should expect the test to fail, so “Test” will return “false.”  This skips the “Commit” operation and checks “TestCodeOnly.”  The “TestCodeOnly” step should return “true,” as we only changed test code, and this prevents the execution of the “Revert” step, which would throw away our changes.

This is the “Red” step of the “Red, Green, Refactor” cycle.  We have one failing test.

Now, as typical of TDD style, we write the minimum amount of code needed to make the test pass.  Then we evaluate the “( Test && Commit ) || TestCodeOnly || Revert” expression again.  We certainly hope that the xUnit “Test” run succeeds, which will result in an automatic “Commit” of our changes.

But if the xUnit “Test” run still fails, then “Test” will return “false.”  Because of this failure, the system will skip the “Commit” step and evaluate the “TestCodeOnly” check.  But this will return “false,” as we did change non-test code, in an attempt to make the test pass.  So the system goes on to run the “Revert” script, which throws away the failing test that we just wrote, and our quick simple attempt to make it pass.  At this point, if we run all the tests again, we will find that they all pass.

It can be frustrating to have the system automatically discard all your changes with a “Revert.”  But doing this does encourage us to “take smaller steps” in developing software, so that we do not risk losing too much.  And this approach has been shown, in practice, to be quite effective.

Example

Let’s suppose that we would like to implement a Fibonacci function in Java using Test-Driven Development, with the “( Test && Commit ) || TestCodeOnly || Revert” expression workflow.

Let’s start with a very simple JUnit test:

import junit.framework.TestCase;

public class FibonacciTest extends TestCase {

    public void test0() {
        assertEquals(0, Maths.fib(0));
    }

}

It doesn’t compile, as there is no “Maths” class, so let’s not run the expression yet.  I’ll let my IDE help me generate the “Maths” class and the “fib” function.  And I’ll put in the simplest possible implementation that will pass the JUnit test above.  I want the test to pass, to get the “Commit” instead of a “Revert” of the code I’ve just written.

public class Maths {
    
    public static long fib(final int index) {
        return 0;
    }

}

I run the “( Test && Commit ) || TestCodeOnly || Revert” expression, and because all the JUnit tests pass, “Test” returns “true,” and “Commit” saves my changes.

Let’s add another test, in the FibonacciTest class:

    public void test1() {
        assertEquals(1, Maths.fib(1));
    }

Evaluating the “( Test && Commit ) || TestCodeOnly || Revert” expression, I find that “Test” returns “false,” because this new test fails, but “TestCodeOnly” returns “true,” because I only made changes to the FibonacciTest class.  I did not change even a single character in the “Maths” class.

Now I need to make the simplest change possible in the “fib” function that will make all of the tests pass.  And I’d better do it right, or I’ll lose my change and my new test!  So, admitting that I’m kind of evil, I “take the lazy way out” and make a one word change to this:

    public static long fib(final int index) {
        return index;
    }

Evaluating the “( Test && Commit ) || TestCodeOnly || Revert” expression, I find that the tests pass, so it commits my changes.

Here’s another test:

    public void test2() {
        assertEquals(1, Maths.fib(2));
    }

Evaluating the “( Test && Commit ) || TestCodeOnly || Revert” expression, I see that the new test fails, but the “TestCodeOnly” check prevents the “Revert.”

I’ll fix it with something that returns the two possible values:

    public static long fib(final int index) {
        return (index == 0) ? 0 : 1;
    }

Evaluating the “( Test && Commit ) || …” commits.

Add a test:

    public void test3() {
        assertEquals(2, Maths.fib(3));
    }

This test fails, but does not cause a Revert.

I’ll do something crazy to make it pass:

    public static long fib(final int index) {
        return index + ((index < 2) ? 0 : -1);
    }

I don’t like how this code “looks,” but it does pass the tests, so I get a Commit.  So this would probably be a good time to refactor.  Something closer to the definition of the Fibonacci sequence would probably be easier to read and maintain.  So I’ll add this:

    public static long fib(final int index) {
        switch (index) {
            case 0:
                return 0;
            case 1:
                return 1;
        }
        return index + ((index < 2) ? 0 : -1);
    }

This passes and commits, so I’ll simplify it to this:

    public static long fib(final int index) {
        switch (index) {
            case 0:
                return 0;
            case 1:
                return 1;
            default:
                return index - 1;
        }
    }

I’m going in the direction of making it recursive, for the “default” case, but let’s add a test to justify adding that complexity:

    public void test4() {
        assertEquals(3, Maths.fib(4));
    }

Oh; it passes and commits.

Let’s try adding another test:

    public void test5() {
        assertEquals(5, Maths.fib(5));
    }

This test fails.

I still want to cheat a bit, like I did before:

    public static long fib(final int index) {
        switch (index) {
            case 0:
                return 0;
            case 1:
                return 1;
            default:
                return index - ((index < 5) ? 1 : 0);
        }
    }

This passes, but has a confusing and annoyingly complex expression.  Is “return index – ((index < 5) ? 1 : 0);” more or less complex than “return fib(index – 1) + fib(index – 2);”?  I’m going to say that the latter is easier to understand, so I refactor to it:

    public static long fib(final int index) {
        switch (index) {
            case 0:
                return 0;
            case 1:
                return 1;
            default:
                return fib(index - 1) + fib(index - 2);
        }
    }

Tests still pass, so it commits.  And I have a simple and easy to understand implementation.  It’s slow, but we can deal with that with a bit more refactoring, should we decide that this is important.

Advertisements

Authority and Responsibility are the same thing

September 20, 2018

Inspired by Ron Jeffries‘ “Manager Responsibilities” article, I responded…

I like that you gave the word “Responsibilities” such prominence. While in some ways I have been quite critical of “management” lately, I think that my issue lies more with the question of authority and responsibility.

I think that…

Authority and Responsibility are the same thing.

And that this is important.

I think that those who try to delegate or impose responsibility on others, while denying them the necessary authority to act are frauds.

And I have seen quite a few people over the years misusing agile concepts to do that. “This is SCRUM.” they say, “And you take full responsibility for your commitment to deliver all these things by the deadline (… regardless of anything I or others do to you). This is ‘your commitment(that I’m imposing on you).” And (sometimes) then, they actively sabotage your efforts and yet still insist on holding you “responsible” for “your commitment.” I find such behavior fraudulent. You give both or you take both. You can’t give one and keep back the other.

Further…
To clarify…

If you use your authority, then you are taking responsibility.

When you intervene and tell your people to stop doing this, and to do that instead… When you tell people to stop doing things the way they think is best, and do things your way instead… Then you have used your authority to override others. And by doing so, you have taken responsibility for the results. No matter how much you might try to deny it. No matter how much you might try to appeal to “their professionalism” or “their job title” as reasons why they should give up everything and possibly “do the impossible” to “deliver on their commitments,” you have used your authority in ways that affect the outcome, and so you are responsible for that.

When I am working as a leader or manger, I strive to keep that in mind: That it’s impossible to delegate responsibility without also delegating authority. And when I use my authority, I am taking responsibility.

I have found that that works well. And I wish that others would also keep that in mind, and do the same.


My Experiences with Requirements Traceability

August 3, 2011

With some positive feedback on my post on the Yahoo Test-driven Development group, I thought I’d post my comments here, to make them more accessible:

— abrar arshadwrote:

Yeah I know requirements traceability has always been related to formal requirements specification and in agile with don’t have formal documentation. … For instances, how do you know where to make changes if your client wants you to change a feature which already has been developed. There is a chance that changing one feature might effect the others as well. …

I was involved in Document-Driven approaches for quite a few years before joining the XP community. “Requirements Tracability” was always a promise of the document-driven approach, used in part to justify its high cost. But honestly, I have never seen it deliver as promised:

Requirements tracability advocates say that it will show you where to make a new change, and that it will highlight conflicting requirements. I have never seen this happen in practice.

Consider this example: We have a system that is computing hourly pay in several divisions at several union plants. There is a fair amount of hard-coded conditional code. We just renegotiated the overtime rates for one of the divisions at two plants.

For requirements tracability to be useful, it has to be easier to find the original requirements for these divisions and to trace these requirements down to code than it would be to find the code directly. And to see conflicts, one would have to go from all the requirements that are relevant to the code back to the requirements documents — and then somehow figure out what to do with a whole bunch of requirements statements — to see if they conflict or overlap in any way, and how to resolve the issues.

Generally, in practice, it’s pretty easy to find the relevant code, even without any external requirements documentation. It’s easier to find the code than to trace through a tangled mess of requirement number references.

And to make the change… Add or change tests. Then change the code so that it passes the tests. If there are conflicting requirements, other tests will fail. You’ll look at the other tests and probably learn something. Sometimes it’s a technical issue, easily solved. Sometimes it is a real conflict in the business requirements. In that case, you will probably have to go back to the business requirements and maybe to the people who specify them to resolve the business issue.

So requirements tracability is not only costly and quickly out of date, it turns out to not be very useful. About the only good thing I’ve seen requirements tracability do is to serve as a checklist of all the things the system must do: When they’re all checked off, then you have reason to believe that the system does everything that’s been requested. User stories with automated acceptance tests also do this — with much higher justifiable confidence levels.


Tim Ottinger’s “Technical Debt” card

February 12, 2009

I like Jeff Langr and Tim Ottinger’s “Technical Debt” card, as a suitable reminder of the costs of accumulating technical debt. I posted one in our workspace. Thanks to Tim and Jeff (L.)!