Posts Tagged ‘ThoughtWorks’

Introducing Depth of Test (DOT)

March 18th, 2012

For the last couple of years I have been particularly passionate and vocal on my projects and in the community about the importance of writing tests as close as possible to where the code is written. As a result I have been achieving easier and cheaper to maintain testing pyramids as opposed to expensive and brittle ice-cream cones. My passion stems from all the times that I saw, and wrote myself, test suites which attempted to achieve most of the high level scenario coverage through the user interface. I was one of the passionate advocates of this technique during a ThoughtWorks Technology Radar session where we collected new ideas. Now I am quite satisfied to see that the technique has recently been added to the adopt section of the latest radar.

ThoughtWorks Radar


Testing at the appropriate level

“The advent of BDD, testing frameworks like Cucumber, combined with browser automation tools like Selenium, has encouraged widespread use of acceptance testing at the browser level. This unfortunately encouraged doing the bulk of testing where the cost to run the tests is the greatest. Instead, we should test at the appropriate level, as close to the code as possible, so that tests can be run with maximum efficiency. Browser-level tests should be the icing on the cake, supported by acceptance and unit tests executed at appropriate layers”

Shallow Depth of Test

I believe that neologistic metaphors, like Ward Cunningham’s Technical Debt, are extremely effective to explain concepts like this. I will explain a real world example and eventually I’ll get to my neologism: Shallow Depth of Tests.

Let’s say, hypothetically, as if I had never worked on one of these, that we have to implement a quote web application that has several business rule validations and if the details provided are valid, it gives the user a price. From the user’s perspective, the app is pretty much like this:

DOT Depth of Test Black Box System


Let’s keep in mind that we want to avoid testing this system as a black box, so let’s break it down and understand what’s inside. The system’s architecture is explained in more detailed in the image below:

  • Javascript layer communicating to server side using JSON services
  • Controller and mandatory validator
  • Domain model, business rules and pricing calculator
  • Data storage

DOT Depth of Test Architecture

The pricing calculator is a crucial component of this system. It has to be thoroughly tested in order to make sure that it provides the right price for several scenarios.If we decide to test pricing through the user interface, using a tool like WebDriver/Selenium, or any other tool that drives a browser the image below shows all the components that will be visited by these tests, I call these components on focus.

DOT Depth of Test Test Flow

For pricing, there are several scenarios, maybe tens, sometimes hundreds of different combinations of factors that might affect the amount to be paid. This means that we will be visiting those components many times when all we are testing is the pricing calculator. In other words, everything will be on focus, when all we want to be focused is the pricing calculator component.

Here is when I start my neologistic metaphor… In optics, particularly in film and photography, there is the concept of depth of field:

Depth of Field (DOF) is the distance between the nearest and farthest objects in a scene that appear acceptably sharp in an image.”



Therefore, adapting this definition to software testing:

Depth of Test (DOT) is the distance between the nearest and farthest software components that get visited during the execution of a test.”

It is important to point out that the definition mentions a “software component“, which is not necessarily one “class” (OOP), or one “function” (FP). Components are logical entities that performs a small feature of the system. It could be an entire pricing calculator, a business rule validator or a simple string concatenation function. Each system will have its own components with various sizes.

Having defined that, if we want to test the pricing calculator mentioned above, we should keep it on focus and test it at a different level, not through the user interface, in this case a browser. If we do that, we will end up having a Shallow Depth of Test.

DOT Depth of Test Shallow

What I have learned and observed is that the more shallow the depth of tests, the cheaper is it to maintain and also the faster it is to execute them.

Technical , , ,

Future Retro Box

November 15th, 2009

We have noticed on our current project that during retrospectives people usually raise good and bad things from the last days only. At the moment we have a 3 weeks iteration and it’s hard to remember what happened on the first days. I have already used from 1 to 4 weeks iteration and each project has its own reasons for choosing this length. To help the team improve around things that happened during the whole iteration, we have a box called the Future Retro Box, where people can all keep adding notes throughout the iteration.


The reason why the box is called Future is because it will be opened during the next retrospective. The items inside the box will bootstrap the retro wall when people can add other ones.

Make sure the box is in a visible point of the work environment.


Another option would be to have an on going retrospective wall, as suggested by my friend Lachlan. However, this might work better for a co-located team, we are distributed. We tried using a wiki for that… It didn’t work… No one adds comments to the wiki… Why? Perhaps the wiki is a Refrigerator as opposed to a Radiator. The box has been working fine so far. The notes are not visible, but the box itself is.

Technical , , ,

Roles and Hand-Offs (Jason Yip)

August 5th, 2009

My colleague Jason Yip posted some very interesting thoughts on roles and hand-offs and I thought perhaps I could express them with some images… As I really like images better than words.
Jason mentioned that usually this is what happens:

Waterfall Project


Some Agile projects

They miss the point by assuming this is enough:


Ideally, this is what we want


“We’re not just re-ordering hand-offs, we actually want to remove them.” (Jason Yip)

Technical , , , ,

How far should we go with estimates

May 4th, 2009

My colleague Dan North facilitated a session today at ThoughtWorks office in Sydney about estimation, creating an MSL (Master Story List), what Scrum calls Product Backlog and which level of granularity should it be.

Dan started the session with a very interesting metaphor: If we want to calculate the area of a country we can always approximate it by comparing it with something that we know how to calculate… A triangle, for example. Dan used the map of the UK, I’m sorry Dan, I couldn’t help myself… I changed it to Brazil… :)


Basically, the area is something in between the areas of MIN and MAX triangles. And it’s very easy to calculate the areas of the triangles, right? How accurate is that? Of course it’s not 100% accurate, but most of the time we don’t need 100% accuracy. We just need to understand the boundaries in order for us to mitigate risks, which have been described as our “fears”.

The estimated range also helps us to make decisions. If a client wants to build an application, for example, and after calculating his current operational manual costs and what the application would make him save, he is willing to pay 1 million. But the estimated range is between 4 and 6 millions, this already helps the client to make the decision of not to proceed with the project.
Always bear in mind what estimates are:

“Estimation is the calculated approximation of a result which is usable even if input data may be incomplete or uncertain.” (wikipedia)

Another pertinent topic during the session was that the Iteration Manager (Scrum Master or XP Coach) should not only be focused on the lower level stories (Product Backlog), but also on the high level project goals and outcomes as well as relating them to the client’s and organization’s needs and purposes.
Stories can be grouped at a A higher level, which we call a theme. Prioritisation and estimation could also be done at these levels.


Some other things that I learned:

Technical , , , ,

ThoughtWorks Xmas Party

December 15th, 2008

On the last saturday, 13th of december, we celebrated ThoughtWorks Xmas in Sydney. It was a wonderful party. Pretty well organized. Congratulation!!!
The theme was Gangster and when we got to the party we were surprised with the fact that we would have chips to play poker, roulette and blackjack.
It was a glamourous night with wine, friends, gambling (fake money) and delicious food.

You can see all the photos here.


Do you want to see more photos?

My picasa

If you really liked the party and want to see other people’s photos:
Bernarda Maia
Professional photographer

Personal , ,

Goal Driven Retrospective

November 23rd, 2008

Retrospectives are usually conducted in order to keep the continuous improvement engine wheels working properly on a project. It is present on most of the agile processes I have worked with, such as Extreme Programming (XP) and Scrum.

The image below describes retrospective that most of us are used to:


On my last project at ThoughtWorks I had the opportunity to introduce to our client, together with Jason, a new type of retrospective, the Goal Driven Retrospective.

Goals and Actions

The Goal Driven Retrospective brings the idea that we should be much more focused on creating actions to achieve common goals shared and agreed within the team and also that will definitely add business value to the client.


Examples of goals:

  • Zero bugs in UAT
  • Productivity x 2: If a similar feature is required, the team should be able to implement it in at least half the time that they took at the first time
  • Improve work environment

Examples of actions:

  • Showcase and run selenium tests on IE to decrease number of browser dependency bugs in UAT
  • Refactor JSP’s and create a DSL to specify exactly what varies from one story to the other
  • Team lunch to improve work environment

It is essential that the person who is facilitating the retrospective, usually the Scrum Master, XP Coach, or Iteration Manager is aligned with the goals of the organization.


In order to prevent conflict of interest, all the goals are defined based on values:

  • Productivity
  • Cost
  • Quality
  • Morale

By following these values the actions will make the team:

  • Faster
  • Cheaper
  • Better
  • Happier

All the actions must be aligned with all the values.

Some examples of actions that would not be acceptable because they go against values:

Where is morale?
What if someone suggests that in order for the team to be twice as productive they need to work 16 hours a day? If you just had that face and thought: “Come on, we know that this doesn’t work…” you are smart aligned with the values, but remember: this is not common sense.

Slow slow slow… what about productivity?
Or what if to achieve zero bugs in UAT we try to test and automate every single possible scenario in the whole application?

The picture below shows the output from the retrospective we had on the client:


I strongly encourage everyone to give it a go and try this new way of improving.

Technical , , , , , , ,

Buying features - prioritizing with poker chips

November 18th, 2008

If you won the lottery and you saw 10 million dollars in your bank account tomorrow, what would you do? You would probably buy a lot of things… right? But actually you do not need everything you would buy, do you? Ok, you would have a lot of spare money, so there’s no problem.

But what if you had only a hundred dollars in your account? What would you buy? You would have to think about it more carefully and prioritize.

The product owner needs to define what is really important for him.
If you ask him what does he want for a 2 years project (10 million) he will say: “I want everything!”, but if you break down the project into small iterations (sprints) and ask him to prioritize what does he want for a small chunk of time, that’s when the important things come up.

A good way to show the product owner that he does not have 10 million available is making him buy features with a limited amount of money.


How does it work?

Before starting the poker chips session, we need a list of estimated user stories. We call it Master Story List (MSL), or Product Backlog. There are many ways of facilitating an estimation session, I like planning poker. But always remember: The team gives the estimates.

Basically you need to define how much a story point costs and give a certain amount of poker chips to the client according to the size of your iteration. And then ask him to buy what he wants for the next iteration.
It is interesting because from my experience as long as he has money, he buys everything, but then when he starts running out of chips, that’s when the real process takes place.


The output will be a list containing the most important user stories at that time.

Technical , , , , ,

Quick Sorting User Stories

November 17th, 2008

We had 8 user stories and we came up with this quick way of sorting them in the office. It was pretty quick… and simple.



Technical , , ,