Estimating stories – How accurate can we be?

So i recently attended a skills matter conference by Linda Rising about estimating stories and how realistically we can be in estimating the size of them which was a real eye opener. A large chunk of it was focused of how our brains actually work and therefore influence how we estimate a chunk of work.The first major point that was made is that we lie to other people and ourselves all the time. This is not just regarding stories, but everything!  We tend to have a rose-tinted view of the world which we means that not only are we over optimistic about our abilities but also the abilities of others. This over optimism then unconsciously gets translated into our estimates. Comments like “oh that’s just a link on a page, that’ll be easy” and “oh we’ve done that before so it’ll be easier this time around” are common place in estimation sessions.

To set the scene of the next point I’ll talk about how people can estimate story sizes. So we use story points of 1,2,4,8. Other companies i know use Fibonacci (1,2,3,5 etc) or “small”, “medium” or “large”. So the next problem is that your imaginary number/size which is relative to other stories of similar size will then have mathematical functions performed on it (e.g. division, subtraction etc) which on an imaginary number, doesn’t hold much value. E.g. you estimate a story as a 4, it then may get split into 2 ‘equal’ stories of 2. Math operations such as this don’t work with an imaginary number created by an unconsciously over-optimistic estimator.

So before you start throwing estimation out of the window there is some hope! The first key point is that do not try to estimate too much, too far in  advance. The result will be inaccurate at best. You’re better off estimating on things you are working on right now, not 6 months down the line. The reason for this is that as you’re working on the story you’re gradually reducing the amount of ‘unknown’ elements to it which means you have a better idea of what work could be still outstanding. The next point to be made is that continually review your estimates as you’re working through the story. Estimates should be a continuous feedback loop to project managers to indicate if your story is on track or spiraling out of control and once your story has been completed, compare it with other stories which had the same estimate.

So to summarize:

  • As humans, we are over-optimistic liars to ourselves so PM’s add a bit of a buffer to the figure we come up with in estimates
  • Don’t estimate too much, too far in advance
  • Continuously review your estimates as you play the story

for those further interested in this topic, here’s the pod-cast: estimation & deception

Is It Ever Ok To Checkin On A Red Build?

We were having an interesting debate at work today as to whether or not it is ok to checkin on a red build. So we had the situation that we needed a release candidate in order for the QA team to test. The build status was red from our CI environment which was preventing a developer with fixes for the release candidate checking in. This had been the state for a couple of hours with situation not being resolved.  So i hear you say “why not just revert back to a green state to allow the developer to checkin on a green build ?” Well now throw into the mix the idea of flakey tests (a flakey test being one which fails sometimes but when you rerun it, it goes green). Now how far do you revert back to ? How would you know where a known good state was without removing all of the flakey tests ? How do we know that our current red build status is a manifestation of a number of flakey tests ?

So the decision was then made for the developer with his fixes to checkin on a red build and low and behold it eventually went green. Which sparked the debate, should we really have a blanket rule saying no checkins on red ?

Pros Of Allowing Developers Checking In On A Red Build

  • Developers do not feel frustrated whilst they wait for hours for a green build
  • Developers can practice small commits often to the CI environment
  • Their checkin may not cause additional failures
  • Stories not related to the release candidate can still proceed
  • Build status may be arbitrarily holding up developers if failure is not genuine

Cons Of Allowing Developers Checking In On A Red Build

  • Release candidate must be green before it can be release to live so may hinder progress
  • Relies on developers confidence that their additional checkin on red will not break anything else
  • Not starting from a known good state from a testing perspective
  • Problem of red build may become compounded if several developers all checkin at the same time
  • It may end up staying red for longer vs holding off for additional checkins

My personal opinion on this that if you have the following situation within your organization then it probably would be acceptable (not great) to check in on a red build:

  • A stable environment with no flakey tests
  • A close knit team not spread among multiple sites
  • No need for frequent release candidates
  • A small build time to allow for quick reverts
  • The ability to diagnose test failures quickly

i would be genuinely interested to see the number of firms which had this. I also realize that the elephant in the room in our organization is to fix the flakey tests to guarantee a build in a genuine failure.

When A Bug Is Not A Bug ?

So I guess we should start we should start by defining what a bug is from an academic perspective. So from the official ISEB stand point a bug is :

A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system.

Source:http://softwareqatestings.com/introduction-to-software-testing/iseb-software-testing-glossary-d-to-l.html

Seems fairly straight forward right ? This is where the real world kicks in. Say you find an a defect on a system and you then show it to a product owner saying “hey I’ve found a defect”. The product owner replies “Oh, it’s not great but we don’t care about that affecting our system.” Still a defect then ? As this is a situation I’ve encountered myself I now have the belief that a defect is something that causes an unexpected behavior and affects something that the business cares about

Here’s another situation. You find an unexpected behavior and show it to a business analyst. The BA replies “Oh, I didn’t know that situation could happen in the system and we don’t have a requirement for that.” In this case it’s a new requirement rather than a behavior

So to summarize, a defect is not always a defect. A defect is something that affects business value which we already have a requirement for its defined behavior

What is Agile and where can I buy it ?

So unfortunately agile is not a product that you can simply buy off the shelf. Agile is an umbrella term for a way of working which incorporates a number of techniques from different disciplines. These are:

  • Extreme Programming (XP)
  • SCRUM

Agile tries to take the best of these and combine it into one business process. So in order to understand some of the ways of working involved in agile, I’ll talk a bit about the above disciplines.

Extreme Programming:

Extreme programming allows software developers to put customer focus at the heart of everything they do. This works by developers working in pairs to deliver solutions which are adaptive to change, incremental and robust. One of the ways this is achieved is by writing automated tests before any functional code is written (test driven development) and the concept of paired programming which is the theory that 2 developers working on the same problem will come up with a better solution than working on their own. More information on this topic can be found here

SCRUM:

Scrum is a rigorous business process which involves developing software in iterations (usually 2 weeks) which are called sprints. The scope of the iteration is defined by an iteration kick off (IKO) where initial objectives of sprint our outlined. Work is broken down into stories which are chunks of deliverable functionality which have value to the business.

Every day there is a progress report from everybody (called a daily SCRUM) which outlines:

  • what they have been working on yesterday
  • what they are going to work on today
  • any blockers to their work

more information on SCRUM can be found here

So Agile takes the best of these in order to produce the Agile Manifesto:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

So what does this actually mean for a business ?

Individuals and interactions over processes and tools: So this is all about communication and improving communication between your different teams (Project Managers, Business Analysts, Quality Assurance, Developers and UX designers). So to tackle communication issues we use a number of techniques:

  • we have an open plan office so everyone can see each other
  • pairing happens across multiple disciplines (e.g. UX & Dev)
  • We all use chat clients to talk to people in remote locations
  • We have video conferencing technology to talk to people in remote locations
  • We all use the same tools when we work on stories

Working software over comprehensive documentation: Now this does mean stop writing down everything you do as this will get you nowhere. it means writing down stuff that people are actually going to read or care about in order to deliver software. So we don’t have 200 page requirement documents or hugely complicated architecture diagrams because of 2 reasons:

  • People don’t read them
  • They change so fast that it’s pointless keeping them up to date

So what’s the alternative? Break chunks of work down into small bits of functionality that delivers value to the business. An example could be: “As a user i want to search for makes of cars from the Homepage”. This would then be broken down into a list of actions that a user could do. Now this means that the story has all the requirements captured in order to deliver it to live without updating and editing a massive requirements document. The benefit of doing this is that a story becomes more adaptable to change

Customer collaboration over contract negotiation: This boils down to working with the customer constantly to deliver something they actually want rather than showing them something at the end of 6 months and it’s all wrong. We do this by having product owners who we showcase stories as soon as we feel they are completed. This means that when software goes live they have already seen a series of interacting parts beforehand

Responding to change over following a plan: I touched on this point earlier about making stories adaptable to change. One thing that i can guarantee is that if you have a large requirements document up front with very little interaction with product owners whilst you are building it is that by the end of it you will have delivered something they don’t want. Requirements change all the time and Agile allows you to be adaptable to change by having stories which deliver small amounts of functionality. So if a requirement changes, no problem. If possible including in the story you are already working on. If the change is too large then raise a new story to deliver that functionality next time

Ok. So How do I start doing Agile?

So make no mistake, Agile is a big change for organizations. It is a different way of working in order to deliver business value and it will take time. One option is to use consultancy companies who specialize in making companies Agile. This is great but can be expensive. Another option is attend free talks on Agile and gather evidence to drive change yourself. With any large change, start small and work your way up

Will Agile make me deliver software faster ?

The jury is out on this one. Agile is no silver bullet and you do have to put the time in to make it work for you. The great thing about Agile is that you can customize to your business need. Personally, i feel that it might not make you go faster (versus something like a waterfall model approach) but it will deliver higher quality software which in turn can improve your time to market.

Summary

So to summarize, Agile is a way of working which uses techniques from different disciplines in order to tackle the problem of developing software. It can be customizable to your needs but you need to put effort into it if you want to get benefit from it

And So It Begins!

So i finally decided to create a blog on software testing, agile practices I’ve used and continuous integration. The aim of this site is out put my experiences that I’ve learned into one place for others benefit from. My only disclaimer is that i don’t have all the answers and i can only go on what I’ve learnt so watch this space!