Archive

Posts Tagged ‘TDD’

Introducing Depth of Test (DOT)

March 18th, 2012

For the last couple of years I have been particularly passionate and vocal on my projects and in the community about the importance of writing tests as close as possible to where the code is written. As a result I have been achieving easier and cheaper to maintain testing pyramids as opposed to expensive and brittle ice-cream cones. My passion stems from all the times that I saw, and wrote myself, test suites which attempted to achieve most of the high level scenario coverage through the user interface. I was one of the passionate advocates of this technique during a ThoughtWorks Technology Radar session where we collected new ideas. Now I am quite satisfied to see that the technique has recently been added to the adopt section of the latest radar.

ThoughtWorks Radar

 

Testing at the appropriate level

“The advent of BDD, testing frameworks like Cucumber, combined with browser automation tools like Selenium, has encouraged widespread use of acceptance testing at the browser level. This unfortunately encouraged doing the bulk of testing where the cost to run the tests is the greatest. Instead, we should test at the appropriate level, as close to the code as possible, so that tests can be run with maximum efficiency. Browser-level tests should be the icing on the cake, supported by acceptance and unit tests executed at appropriate layers”

Shallow Depth of Test

I believe that neologistic metaphors, like Ward Cunningham’s Technical Debt, are extremely effective to explain concepts like this. I will explain a real world example and eventually I’ll get to my neologism: Shallow Depth of Tests.

Let’s say, hypothetically, as if I had never worked on one of these, that we have to implement a quote web application that has several business rule validations and if the details provided are valid, it gives the user a price. From the user’s perspective, the app is pretty much like this:

DOT Depth of Test Black Box System

 

Let’s keep in mind that we want to avoid testing this system as a black box, so let’s break it down and understand what’s inside. The system’s architecture is explained in more detailed in the image below:

  • Javascript layer communicating to server side using JSON services
  • Controller and mandatory validator
  • Domain model, business rules and pricing calculator
  • Data storage

DOT Depth of Test Architecture

The pricing calculator is a crucial component of this system. It has to be thoroughly tested in order to make sure that it provides the right price for several scenarios.If we decide to test pricing through the user interface, using a tool like WebDriver/Selenium, or any other tool that drives a browser the image below shows all the components that will be visited by these tests, I call these components on focus.

DOT Depth of Test Test Flow

For pricing, there are several scenarios, maybe tens, sometimes hundreds of different combinations of factors that might affect the amount to be paid. This means that we will be visiting those components many times when all we are testing is the pricing calculator. In other words, everything will be on focus, when all we want to be focused is the pricing calculator component.

Here is when I start my neologistic metaphor… In optics, particularly in film and photography, there is the concept of depth of field:

Depth of Field (DOF) is the distance between the nearest and farthest objects in a scene that appear acceptably sharp in an image.”

wikipedia

440px-Depth_of_field_diagram.png

Therefore, adapting this definition to software testing:

Depth of Test (DOT) is the distance between the nearest and farthest software components that get visited during the execution of a test.”

It is important to point out that the definition mentions a “software component“, which is not necessarily one “class” (OOP), or one “function” (FP). Components are logical entities that performs a small feature of the system. It could be an entire pricing calculator, a business rule validator or a simple string concatenation function. Each system will have its own components with various sizes.

Having defined that, if we want to test the pricing calculator mentioned above, we should keep it on focus and test it at a different level, not through the user interface, in this case a browser. If we do that, we will end up having a Shallow Depth of Test.

DOT Depth of Test Shallow

What I have learned and observed is that the more shallow the depth of tests, the cheaper is it to maintain and also the faster it is to execute them.

Technical , , ,

Testing Pyramid - A Case Study

March 5th, 2012

Test automation is prevalent in the software development community. Practices like TDD and BDD are widespread and applied almost unquestionably. However, several organisations have struggled in attempting to scale automated test suites, which very often become slow, brittle, non-deterministic, unreliable and difficult to maintain.

One common issue reported by many teams and that I have also experienced many times is the inverted testing pyramid.

A year ago I joined a project that was going in that direction. I could see us making the same mistakes again. If we had kept going we would have ended up with the slow and hard to maintain, melting ice-cream cone. However, this time we tried a different approach. Our test strategy was heavily based on the concept of Shallow Depth of Tests, which means testing the code as close as possible to where it is written using preferably either unit or integration tests. We only automated high level test journeys at the UI level.

Now, one year later, we have a stable and fast build, which gives us an extremely high level of confidence.

Here is our project’s testing pyramid, of which we are very proud:

Testing Pyramid Fabio Pereira

Some observations:

  • Only 12 tests through the UI take 13 minutes to run
  • 1748 unit tests take only 1 minute
  • 273 JavaScript unit tests take less than 1 second. Treating JS as a 1st class language helped us as well

Feel free to share your testing pyramid as well. And always keep an eye on it… It can make a big difference to your project.

Technical ,

BDD with Scenario Code DSL – Sometimes you don’t need a tool

May 28th, 2010

My previous project was distributed across cities and we had almost 30 people involved. Communication and the understanding of what needs to be done are crucial for the success of the project.

One of the big challenges was to make sure that the business and the technical people shared the same domain Ubiquitous Language. To ensure that, we started using JBehave to define the acceptance criteria of the stories. Half way through, there was a big change which required many changes in the scenarios…

Some issues that we found:

  • The scenarios are text files and extremely hard to refactor.
  • We had to change the text files (scenarios) and the Java files and keep them in sync
  • Too much duplication
  • When writing the scenarios we had no such thing as auto completion
  • When running the scenarios, we could not run 1 scenario at at time, but the whole file

 

Slide 28052010 111155 AM

The solution was to abstract our own code with a DSL layer and use it for documentation purposes. Initially we called this JMisbehave because the initial intent was to generate a read only version of the code that we could show to non-technical people and agree on acceptance criteria, as well as keep an executable documentation of the application. Code Humaniser was another suggested name…

 Fullscreen capture 28052010 42326 PM 

Eventually we realised that the code itself was sufficient and we ended up not using the code that generated the read only version of it. Nevertheless it’s still useful in many environments. Many times we showed the code to non-technical people and they were quite delighted to see how straight forward it was to read and to understand. Moreover, it was expressing the Language of the Business. An interesting thing is that our colleague Dan North, who wrote JBehave, came to our project one day to facilitate a retrospective and we told him the whole story. He quite liked what we had done and was quite complimentary about the fact that we had understood the concept of DDD and BDD, and the fact that it’s not about the tools. It was actually his recommendation to use the_underscore_notation instead of theCamelCaseNotation, we had some discussions about that, but that will be described on another post.

Behind the Scenes - The Scenario Code DSL Implementation

James Barritt saw another post of mine with some of our Scenarios Code DSL examples and suggested that I wrote a post explaining what was happening “behind the scenes”.  Here is a high level diagram of how we separated the layers:

 Fullscreen capture 28052010 43759 PM

Some code:

Scenarios Code

public class Location_Scenario extends BaseScenario {
  public void should_default_risk_address_when_insured_address_is_valid_for_risk() {
    given(the_broker. has_started_a_home_quote());
    when(the_broker. selects_a_valid_location_for_risk_address());
    then(the_policy.has_home_risk_location_and_risk_address_as_same());
  }
}

public class BaseScenario {
  protected Broker the_broker;
  protected Policy the_policy;
  (...)

  protected <T extends DSL> T given(T dsl) {
    return dsl;
  }

  protected <T extends DSL> T when(T dsl) {
    return dsl;
  }

  protected <T extends DSL> T then(T dsl) {
    return dsl;
  }

  protected <T extends DSL> T and(T dsl) {
    return dsl;
  }

}

DSL Code

public class Broker extends DSL {
  public FieldEnterer enters;
  public Policy the_policy;

  public Broker blah() {
     // call implemenation
    return this;
  }
}

public class Policy extends DSL {
  public FieldEnterer enters;
  (...)
  public Policy blah() {
     // call implemenation
    return this;
  }
}

This was a project with lots of people coming in and out all the time and every time someone new would join, the time taken to understand the code and relate it to the business terminology was extremely reduced by the way our tests were written.
You can do the same on your project, write your tests using the language of the business.

Technical , , ,

TTDD - Tautological Test Driven Development (Anti Pattern)

May 27th, 2010

One of the advantages of being a consultant is getting to see different environments and being able to visualise and identify patterns and anti-patterns.

“An anti-pattern is a pattern that may be commonly used but is ineffective and/or counterproductive in practice” http://en.wikipedia.org/wiki/Anti-pattern

As a first step, let’s describe TTDD – Tautological Test Driven Development as an anti-pattern of TDD (TestDrivenDevelopment). In fact TTDD is a big enemy of the combination BDD (BehaviourDrivenDesign) and TDD.

I still have the vivid memory of the day that I was pairing with Dave Coombes and he mentioned that one of the tests I had written was tautological.

Tautological: - needless, redundant repetition of an idea - repetition of same sense in different words; - the phrase “a beginner who has just started” is tautological

I did not quite understand at first why the test was tautological, i.e. needless, redundant. As a perfect leader and coach that Dave is, he helped me rewrite the test in a different way and that was when I finally understood what he had meant by that.

Example

The example below is simplified version of the one that Dave and I were using at the time. In this example, we want to test the class CarRepository that interacts with 2 collaborators as shown below:

Fullscreen capture 26052010 122651 PM

In order for CarRepository to retrieve all the cars from CarService it needs a ServiceHeader that is provided by ServiceHeaderFactory. For the purposes of this example, I’ll show the implementation of CarRepository:

public class CarRepository {
  private ServiceHeaderFactory serviceHeaderFactory;
  private CarService carService;

  public CarRepository(ServiceHeaderFactory serviceHeaderFactory,
                    CarService carService) {
    this.serviceHeaderFactory = serviceHeaderFactory;
    this.carService = carService;
  }
  public Cars findAll() {
    ServiceHeader serviceHeader = serviceHeaderFactory.create();
    return carService.findAll(serviceHeader);
  }
}

Tautological Test

 

The test below is similar to the one I wrote that we described as tautological:

@Test
public void shouldRetrieveCarsFromCarServiceUsingTheRightServiceHeader() throws Exception {
  // GIVEN
  ServiceHeader serviceHeader = new ServiceHeader();
  ServiceHeaderFactory serviceHeaderFactoryMock = mock(ServiceHeaderFactory.class);
  when(serviceHeaderFactoryMock.create()).thenReturn(serviceHeader);
  CarService carServiceMock = mock(CarService.class);
  CarRepository carRepository = new CarRepository(serviceHeaderFactoryMock, carServiceMock);

  // WHEN
  carRepository.findAll();

  // THEN
  verify(carServiceMock).findAll(serviceHeader);
}

Why is this test called Tautological?

 

One of the definitions of a tautology is: “repetition of same sense in different words”

If you look carefully at these 2 lines of implementation and 2 lines of test:

Test

  • when(serviceHeaderFactoryMock.create()).thenReturn(serviceHeader);
  • verify(carServiceMock).findAll(serviceHeader);

Implementation

  • ServiceHeader serviceHeader = serviceHeaderFactory.create();
  • return carService.findAll(serviceHeader);

They are almost “equivalent”. When we write tests this way, most of the time if the implementation changes, we end up changing the expectations of the test as well and yeah, the tests pass automagically. But without knowing much about its behaviour. These tests are a mirror of the implementation, therefore tautological.

 

The test below verifies the same class CarRepository, but as a black box test, i.e. it does not test the interactions, but the output of the repository. Look at the assertion.

@Test
public void shouldBeAbleToRetrieveCars() throws Exception {
  // GIVEN
  Cars carsFromService = new Cars(new Car("Ferrari"), new Car("Porsche"));
  CarRepository carRepository = givenARepositoryAttachedToACarServiceWithCars(carsFromService);

  // WHEN
  Cars carsFromRepository = carRepository.findAll();

  // THEN
  Assert.assertEquals(carsFromService, carsFromRepository);
}

private CarRepository givenARepositoryAttachedToACarServiceWithCars(Cars carsFromService) {
  ServiceHeader serviceHeader = new ServiceHeader();
  ServiceHeaderFactory serviceHeaderFactoryMock = mock(ServiceHeaderFactory.class);
  when(serviceHeaderFactoryMock.create()).thenReturn(serviceHeader);
  CarService carServiceMock = mock(CarService.class);
  when(carServiceMock.findAll(serviceHeader)).thenReturn(carsFromService);
  CarRepository carRepository = new CarRepository(serviceHeaderFactoryMock, carServiceMock);
  return carRepository;
}

There is still a big effort to setup the mocks and inject them into the repository. This has been extracted to the method givenARepositoryAttachedToACarServiceWithCars. However, all these mock setups make it harder to understand the intent of the test, the responsibility of the class that we are trying to test. When this happens I usually tend to use stubs instead of mocks. Here is the same version of the tests, but using Stubs:

@Test
public void shouldBeAbleToRetrieveCars() throws Exception {
  // GIVEN
  Cars carsFromService = new Cars(new Car("Ferrari"), new Car("Porsche"));
  CarRepository carRepository = givenARepositoryAttachedToACarServiceWithCars(carsFromService);

  // WHEN
  Cars carsFromRepository = carRepository.findAll();

  // THEN
  Assert.assertEquals(carsFromService, carsFromRepository);
}

private CarRepository givenARepositoryAttachedToACarServiceWithCars(Cars carsFromService) {
  ServiceHeaderFactory serviceHeaderFactory = new ServiceHeaderFactoryStub();
  CarService carService = new CarServiceStub(carsFromService);
  CarRepository carRepository = new CarRepository(serviceHeaderFactory, carService);
  return carRepository;
}

 

download ttdd example

It is not a rule, but I find that tautological tests have more mock setup. When we start to think about the collaborators as stubs, the tests become more behavioural.

 

What are the common characteristics of a Tautological Test?

  • Asserts more interactions with collaborators than the outputs;
  • It doesn’t really test the behaviour of the class, but only its implementation
  • The test is too white box
  • Too much mock setup deviates the intent of the test
  • Adding a new feature or changing an existing one requires changing mock expectations

Technical , ,