Note that this no longer represents my thinking, but is maintained for posterity and ridicule.

Agile Planning and Estimating covers the activities involved in actually planning how the software being developed is going to be built, splitting up the development into manageable pieces and deciding a schedule for the release.  This part of my Agile development guide is based on the excellent book by Mike Cohn, Agile Estimating and Planning.

The following picture, taken from http://www.ebgconsulting.com/WorkshopsForAgileProjects.jpg, provides a clear overview of the activities completed during an agile project.

Release Planning

A release plan is the high level plan that is used to identify what needs to be completed and what time frame is required to complete enough before a releasable product is achieved.  The release plan is also the milestone that the team will be working towards over the course of the project and is used in determining overall project health.  With this plan the team has a clear goal of not only the project expectations, but also the expectations of the team; it provides overall goals.

A project must have objectives, this is most often financial but can be evaluated in a number of different ways.  I’ll write another post of producing a business case  to detail the steps that can be taken in evaluating and justifying a project.  Realistically, a project should deliver value in the most efficient way possible.  It may be the case that realising 80% of the total value in only 20% or 50% of the time is far more desireable.  Identifying what themes are required to deliver the desired outcomes is a critical part of this stage.

A project may be feature driven or time driven.  In the former case, the team will derive a time scale by summing the points for all of the selected stories, dividing by the teams velocity (To give the number of required iterations) and then multiplying by the length of the iterations.  This will generate the expected time to completion for the project.

For a time driven project, you should calculate the amount of available story points and then proceed add stories until the allotted time has been filled.  You can determine available story points by dividing the time frame by the iteration length (To give available iterations) and then multiplying by the velocity.

In order for a sponsor to identify what the objectives, constraints and expectations of a project are, the team should have estimated any themes or stories that may have a reasonable chance of being included in the release.

With the completion of these activities, the project will have a justified reason for undertaking it, expectations of various stakeholders and both time and cost estimates.

The final stage is to sketch out at least the first iteration, and possibly an additional 1 or 2.  Planning further than this can be wasted effort as when new knowledge about the product and project is obtained, future priorities will undoubtedly change.  It is often best to simply keep the next iteration planned and ready to go, the following 1 or 2 roughly planned and the remaining stories in the backlog.

Iteration Planning

Iteration planning takes place at the very start of any iteration and will completed by the development team, product owner and any other appropriate stakeholders.  The process follows much of the same system as that of release planning but the planning horizon is reduced to just the next iteration.

1. Updating

At the start of an iteration plan, it is appropriate to first ensure that the estimates for the remaining tasks are up to date and appropriately prioritised.

2. Goal Selection

Next, the team will select a goal that generally represents what it is that they would like to achieve during the iteration.  Something reasonably abstract is fine as long as it represents a cohesive goal that the team can align themselves with.  Stories will be selected for the iteration based on priority, but also in accordance with how well they align with this goal.

3. Choosing Stories

The team and product owner will choose stories based on priority and alignment with the iteration goal until they stories they have selected have an aggregate point cost as close to their estimated velocity (Without exceeding it) as they can.

4. Decomposition

There may be large stories or themes (sometimes simply called ‘features’) pulled into the plan that require decomposition.  This is most often the case where placeholder items were previously added that didn’t require further exploration or estimation at the time.  For these, the team should decompose the item into appropriately sized smaller stories that still represent complete functionality and good business value.

The next stage in iteration planning is to decompose the selected stories into technical tasks and estimate them.  These tasks represent the mechanical activities that need to be completed in order to achieve completion of the story.  It is appropriate to estimate in hours at this point as the team is more certain of the requirements at this stage in planning.  Do not be tempted to derive correlation between story point and task hours, they are very separate measurements that are used for completely different activities.  Story points provide a high level view of features and effort, technical tasks are there to provide a fine grained list of activities to complete.  Story points are for the owner, tasks are for the team.

Estimating Tasks

Tasks are estimated in terms of ideal time, this is the time in which you believe it would take if you could focus on this task and nothing but this task.  Do not be tempted to factor telephone conversions and come up with an effectiveness figure to base it on.  Simply use the Ideal Time.

These estimates should be group efforts.  Although different people will take different amounts of time for a particular task, it is not known who be completing the task at this point so a general consensus should try to be reached.  Agile is a team approach.

Task Size

A developer should generally try and complete one task per day, this ensures that the developer can be focused on a single task rather than having to context switch and provides a clear, attainable goal on a day to day basis.  This also allows the daily scrum to be as concise as possible.

Iteration Planning Guidelines

• Do not include tasks in the iteration plan that do not represent project value, things such as answering emails or attending meetings for other projects do not contribute to the value of the product.

• Be as specific as you can when starting to estimate, break the stories down in such a way that you feel confident that you can walk away and by completing those tasks, the story will be done.  Consider the time to write unit tests, any workshops that are scheduled for the project.

• Bugs should be included in the project in the same way that stories are.  They need to be estimated and prioritised by both the team and owner.  A story that delivers huge value and takes little time is of infinitely more importance than a fringe bug that would take days to fix and deliver very little value.

• Be mindful of dependencies when adding tasks or stories, ensure that stories are as independent as possible but make sure that you will have a clear run at tackling a problem.  A small, low importance task or story that needs to be completed before the start of a critical story needs appropriate planning.

• If there is uncertainty surrounding an estimate or story, schedule a small research spike that will have the goal of providing clarity.  This is a task of an hour or so in length whose primary objective is to provide greater certainty in making an estimate for a story.  Feel free to include a placeholder value for the task or story if it is to completed in the current iteration, but be aware of it’s coarse estimate and try to allocate spikes in previous iterations wherever possible.

• It is appropriate at this point to sketch out rough designs or UI’s in order to clarify details with the team or product owner, but these artefacts should not be considered concrete solutions.

Choosing an Iteration Length

Most iterations fall between between 2 and 4 weeks, I prefer shorter terms in order to improve feedback and reduce the planning horizon.  For projects that are less than 4 months, it is highly recommended that an iteration length of two weeks be used.  For longer projects, it is possible to use up to 4; however, when choosing a length, be sure to factor in the importance of how often you want to be benchmarking progress.  Remember, having a single 4 week iteration does not take much less managerial time than having a pair of 2 week iterations.  Think about the following factors when determining a length:

• The length of the release being worked on
• The amount of uncertainty
• The ease of getting feedback
• How long priorities can remain unchanged
• Willingness to go without outside feedback
• The overhead of iterating
• How soon a feeling or urgency is established.

After choosing a length, it is important to maintain consistency, the iterations are the heartbeat of a project and a consistent beat helps to ensure an effective rhythm.

Estimating Velocity

Estimating velocity can be very difficult, especially if the team or domain is new.  There are generally three options for determining what this value should be.

Historical Values

Consider the following questions:

• Is the technology the same?
• Is the domain the same?
• Is the team the same?
• Is the product owner the same?
• Are the tools the same?
• Is the working environment the same?
• Were the estimates made by the same people?

If you answered yes to most or all of these questions then the chances are that you should be able to use historical values from a previous project.  Getting to use these values is most common when the same team is working on a new version of a previously completed project.  If you answered no to some or all, then it is probably worthwhile considering an alternative.

Run an Iteration

The ideal, and preferred, way of determining an initial iteration velocity is to actually run an iteration.  The team can then use the observable figure as an estimate for future work.  If you have previously been working on traditionally managed projects, you could consider this time as a shortened version of the detailed requirements gathering or specification generation time.  Do not consider this a time to perform those same activities however, this first iteration is a real sprint to deliver value, but with the side goal of getting a velocity estimate.

Make a Forecast

If the team does not have empirical evidence and running an iteration isn’t an available option, then the team can consider making a forecast.  The steps are as follows:

1.    Estimate the hours available

For each team member, estimate the amount of available hours that they have each day to work on the project.  For most people assigned full time to a project, this is usually somewhere between 55% to 70%.  Do not spend time factoring in holiday or sickness time, it is usually wasted effort.  Sum the total hours of each member.

2.    Estimate the time available

Multiply the hours available per day by the available days in an iteration.

3.    Expand the stories

Arbitrarily choose stories of varying length and expand into hourly estimated tasks.  Keep following this procedure until the available hours have been consumed by the expanded stories and tasks.  Try and choose a variety of different length and uncertainty in the stories.  By summing the total value of the selected stories, you have a rough estimate of the initial team velocity.  To further indicate the uncertainty in this estimate, apply a range by multiplying the figure by the low and high end multipliers.

Updating a Velocity

The more iterations that a team runs, the more accurate the estimation will be as it is updated.  It is advised that the team takes a rolling average of the previous 3 iterations.  Initially, the range of uncertainty will be quite high, but as the team starts to get into a rhythm and feeds back empirical evidence into the estimate, the uncertainty should be reduced in a way similar to the following table.

Buffering for Uncertainty

For projects with a high degree of uncertainty or where were missing feature/time estimates would have a larger than usual adverse affect on the business, estimates can be buffered to reflect the uncertainty inherent in any estimation activity.

Feature Buffers

Features in a project are usually of varying importance to the customer, by including items that are not ‘must’ or ‘should’ haves and then planning the project with the aim of delivering all of the functionality allows for flexibility.  To do this, the customer will identify which features the project absolutely must deliver, and the continues to choose a further 25% to 40% of additional work that represents value but is not critical.  The project is estimated using all of the stories, but should it become apparent that the project will overrun, those additional features can be removed from the schedule in order to bring the plan back on track.  What percentage of additional features to include is dependant of the amount of uncertainty that requires buffering and the size of the project.

Schedule Buffers

To buffer a time in the project schedule, we need some way of representing the level of uncertainty.  This is to stop us arbitrarily applying padding, rather than setting an accurate range.  To get a more accurate representation of the uncertainty, we should start with two different figures.

The mostly likely time, or your current best estimate, should be considered the 50% estimate.  This means that there is only a 50% chance of you finishing in the time estimated, and a 50% chance of taking longer.

The second figure is your 90% estimate, this is the figure that represents your 90% certainty in the time it would take to complete a task.  The additional time between the estimates is called a local safety.

Simply summing all of these local safety values and giving that as your buffer is inappropriate as it is highly unlikely that everything will go wrong to that extent.  The benefit of using two estimates is that the ratio of local safety actually represents the uncertainty in that estimate, which allows us to derive the risk associated with each story.  To buffer the project to our 90% certainty estimate, we require a buffer of two standard deviations.

The formula for this is as follows:

2σ = 2 * sqrt( (((w1 – a1)/2)^2) + (((w2 – a2)/2)^2) + …  (((wn – an)/2)^2) )

Simplified to:

2σ = sqrt( ((w1 – a1)^2) + ((w2 – a2)^2) + … + ((wn – an)^2) )

Where:

σ = Standard deviation

w = Worst case estimate (90%)

a = Average estimate (50%)

1, 2, n = Story

This basically just means that you square the local safety for each story, sum the results and take the square root of that answer.

Add this result to your original 50% estimate aggregation for the stories and you have the revised project estimate.  This schedule safety figure (The amount of buffer added) should be added to the end of the project, not distributed over each story.

Example:

Say that you have identified the following stories.

You would complete the following steps.

1. Calculate local safety for each feature
2. Square each local safety
3. Sum all squared local safety estimates
4. Take the square root the total
5. The remaining number is your buffer estimate and should be added to your initial to give a range.

Combining Buffers

To further reduce risk, it is often useful to use both types of buffers.  For example, you could specify that a project will complete:

• All core features
• 0% to 100% of additional features

And will be completed at some point between the original estimate and the padded estimate.  This approach allows the customer more flexibility in determining when the project should end.  The project delivery could be summarised as follows: