For the last couple of years I have been particularly passionate and vocal on my projects and in the community about the importance of writing tests as close as possible to where the code is written. As a result I have been achieving easier and cheaper to maintain testing pyramids as opposed to expensive and brittle ice-cream cones. My passion stems from all the times that I saw, and wrote myself, test suites which attempted to achieve most of the high level scenario coverage through the user interface. I was one of the passionate advocates of this technique during a ThoughtWorks Technology Radar session where we collected new ideas. Now I am quite satisfied to see that the technique has recently been added to the adopt section of the latest radar.

 

Testing at the appropriate level

"The advent of BDD, testing frameworks like Cucumber, combined with browser automation tools like Selenium, has encouraged widespread use of acceptance testing at the browser level. This unfortunately encouraged doing the bulk of testing where the cost to run the tests is the greatest. Instead, we should test at the appropriate level, as close to the code as possible, so that tests can be run with maximum efficiency. Browser-level tests should be the icing on the cake, supported by acceptance and unit tests executed at appropriate layers"

Shallow Depth of Test

I believe that neologistic metaphors, like Ward Cunningham's Technical Debt, are extremely effective to explain concepts like this. I will explain a real world example and eventually I'll get to my neologism: Shallow Depth of Tests.

Let's say, hypothetically, as if I had never worked on one of these, that we have to implement a quote web application that has several business rule validations and if the details provided are valid, it gives the user a price. From the user's perspective, the app is pretty much like this:

 

Let's keep in mind that we want to avoid testing this system as a black box, so let's break it down and understand what's inside. The system's architecture is explained in more detailed in the image below:

• Javascript layer communicating to server side using JSON services
• Controller and mandatory validator
• Domain model, business rules and pricing calculator
• Data storage

The pricing calculator is a crucial component of this system. It has to be thoroughly tested in order to make sure that it provides the right price for several scenarios.If we decide to test pricing through the user interface, using a tool like WebDriver/Selenium, or any other tool that drives a browser the image below shows all the components that will be visited by these tests, I call these components on focus.

For pricing, there are several scenarios, maybe tens, sometimes hundreds of different combinations of factors that might affect the amount to be paid. This means that we will be visiting those components many times when all we are testing is the pricing calculator. In other words, everything will be on focus, when all we want to be focused is the pricing calculator component.

Here is when I start my neologistic metaphor… In optics, particularly in film and photography, there is the concept of depth of field:

"Depth of Field (DOF) is the distance between the nearest and farthest objects in a scene that appear acceptably sharp in an image."

wikipedia

Therefore, adapting this definition to software testing:

"Depth of Test (DOT) is the distance between the nearest and farthest software components that get visited during the execution of a test."

It is important to point out that the definition mentions a "software component", which is not necessarily one "class" (OOP), or one "function" (FP). Components are logical entities that performs a small feature of the system. It could be an entire pricing calculator, a business rule validator or a simple string concatenation function. Each system will have its own components with various sizes.

Having defined that, if we want to test the pricing calculator mentioned above, we should keep it on focus and test it at a different level, not through the user interface, in this case a browser. If we do that, we will end up having a Shallow Depth of Test.

What I have learned and observed is that the more shallow the depth of tests, the cheaper is it to maintain and also the faster it is to execute them.