Stop locking down the Agile project management tool

I have seen this common behaviour at many of the bigger client organisations I have worked with, who are transitioning to organisation wide agile adoption. The company will invest a lot in researching and eventually buying an agile management tool of choice (Greenhopper, Mingle etc...) and then lock it down so that project teams will have minimal access to change how they want to use the tool.

The sheer irony is that these tools are actually written to be as adaptive as possible to the development process followed in the team. These tools, and especially Mingle for one, is written keeping in mind teams that are following agile will evolve their processes as they move forward.

Story statuses and types could evolve a lot. For a team that has just started, they might just have to deal with stories and defects. But once the team starts making releases, they might also have to deal with enhancements which probably has its own workflow. Changes have to be done to the project tracking tool to accommodate this change in processes. These changes are not one off. A team might discover new types of work and also new ways of tracking and progressing through the work on a frequent basis.

The driver for locking down the project management tool is to force teams to use the same set of statuses and processes so that reporting can be rolled up at a program or an organisation level. A lot of times, senior program management tries to come up with an "ideal agile workflow/template" that should work for all teams and then use the tool to impose it on all teams within the program. The result of this is immense frustration within teams while they try to find workarounds to the system that is being pushed on to them. Teams in this state start moving away from the tool that was chosen and end up tracking themselves through more lo fi means (physical card walls etc..). This reflects poorly on the investment made in buying this agile project management tool in the first place.

Here are a few suggestions

* Roll up reports at a fairly higher level of abstraction when reporting for a program. This gives teams some more flexibility in coming up with actual numbers for the KPIs. For e.g. instead of tracking "Velocity in points", track "Velocity". This gives flexibility for one team to use story points and another  which can use days, and another which can count stories :-).

* Don't rollup reports and don't lock down your workflows when the teams are still forming and storming. Let teams reach a stage where their development processes have matured a bit. Until then let each team present a different status report for themselves. Start rolling up once the teams have reached a performing state together.

Remember, every agile team has to adapt to different set of challenges and they can potentially come up with different creative processes to deal with them. For these teams to succeed the supporting systems and tools need to embrace change rather than impose order.

A Distributed Team working out of a Common Backlog

Even though working out of a common backlog might seem the most obvious thing to do, a lot of teams I worked on previously did not do it, or do it that effectively.

Having a common product backlog of stories enforces at the very fundamental level that it is one team spread across multiple locations than two teams.

Benefits

Having a common deck of stories for both locations has the following benefits

Strong levels of trust and collective ownership across people at both locations

Since people across both locations are looking at the code written by each other regularly, it ensures that there is strong collective code ownership. Consistency in coding styles, metaphors, development processes, etc... all fall in place in the long term.

Having a common standup, iteration planning meeting, developer huddles, showcases, retrospectives etc... helps to synch up the team across both locations.

If teams are working on different backlogs, there are always late integration issues along with trust issues when things do not fall in place.

Knowledge distribution across distributed locations

Having stories for a particular feature distributed across both locations ensures that not only people across locations touch/refactor each others code, but it also ensures awareness of features and areas in the business domain across the team.

Allows for team scaling across both locations

Having a cross functional team across both locations who work on the same product backlog provides flexibility to scale the team across regions depending on availability of skill. This model is also very useful in a client-supplier model where the client wants their developers to work with the supplier as well. A good mix of people across both locations ensures that the team can easily scale when needed.

Concerns

However the common backlog model comes with its share of concerns

Very high level of day to day communication across all roles

The basic premise of working with a common backlog is to ensure both locations of the team are always on the same page during their working day. This means a lot of communication via multiple channels such as video conference calls, emails, voice chats etc.

One key factor is the overlap time across both locations. If the overlap time is anything close to half a day then it is advisable that the team at a location has a continuos video stream of the remote team. With the right hardware investment, this can create a very powerful "one team atmosphere" for the overlap time.

If the overlap time is much less (1-2 hours) then a lot of communication has to flow through offline emails and chats. It becomes critical for the team on both sides to communicate the critical pieces of proceedings at a particular location so that the other location is on the same page.

Business and IT stakeholders not being distributed 

Even though the delivery team might be nicely distributed, there is always the common case of the business stakeholders sitting in only one location. This could also apply to the network infrastructure, IT security etc... Because of this, the delivery team in the same location as these stakeholders will end up having a lot more context (e.g. business domain context) and a lot more meetings to deal with than the remote/satellite team which is fairly isolated.

This could cause some uneven knowledge distribution over the long term and should be fixed with adequate rotations across roles within the delivery team.

Conclusion

It will be painful initially to communicate using a common backlog, but dealing with the pain more often (daily) puts the team in a much better place long term than each team going their own way.

Measuring progress

Measuring progress in software development has always been up for debate. There was the waterfall era of Gantt Charts created using MS Project which used to breakdown effort into smaller work items with dependencies. These were easily outdated the very next day they were created.

The Agile community brought in the idea of user stories and doing them in small iterations. We then measured progress using a burn-up or burn-down chart which burnt either story points or ideal days. Soon, we started feeling the pain of the inventory we carried between releases which happened in a span of 3-6 months. This prompted us to measure and limit our work in progress and measure lead and cycle times across the delivery workflow.

Releases started becoming more frequent with technology advances in the cloud, virtualisation and deployment automation. The continuos delivery era came through with examples of people deploying almost everyday to production. Startups really needed this to validate their ideas in the market and course correct themselves based on market feedback. A/B testing became common place and measuring validated learning and cost of delay in deploying a feature made more sense for these new businesses.

Anyone can easily get confused trying to decide how one should measure progress or at least start with, in their project.  There are 2 key variables in making this decision, the kind of relationship IT has with the Business, and the type of product ultimately being built.

So here is the quadrant I am talking about.

Outsourced / Utility (bottom left)

To the extreme left on the relationship axis is outsourcing your IT. Trust levels are fairly low. It could be a new IT vendor who has not delivered for the business. Work is probably happening outside the office or even the country where the business exists.

The extreme bottom of the product axis sit utility/internal applications which are mainly for keeping the business lights on. (Time Sheets, Payroll, Company intranet etc...). There is no market advantage to be gained by these. They are well defined internal business problems.

Start with measuring progress in these cases using effort estimates such as Ideal Days, Story Points, T-Shirt sizes etc... A burn up chart should be a good indicator of where the project stands. Try to work on building enough trust which enables you to move to the right axis in terms of relationship.

Partnership / Utility (bottom right)

Moving right on the relationship axis is only possible by building trust. And that is possible by delivering frequently to the business and also maintaining the people relationships at the right levels. A vendor who is treated as a mere supplier can become a trusted partner for well defined applications and their maintenance as they prove themselves to have the required expertise.

Move progress measurements from effort based units to delivery efficiency based units. Optimise the workflow till production by minimising inventory you are carrying and reducing lead times. A Cumulative Flow Diagram  (CFD) can provide the same view.

Partnership / Strategic (top right)

This is really the sweet spot. This is where the business has engaged IT to build a strategic product which is probably completely new to the market. It could also be a product that is struggling and badly in the need for new ideas. The IT here is a closely trusted partner which the business can rely on.

Given the volatile nature of what needs to be built, the need for agility is the highest here. Being able to deploy a new feature and rolling back as quickly as it was deployed, if hypothesis fails, is equally important.

Measure progress here using business outcome based units such as validated learnings and measuring cost of delay of a feature and the impact it has on the business. 

Partnership / Outsourced (top left)

This hardly ever happens where a strategic business need for technology is outsourced to a less trusted supplier !

Build enough trust before you can stop estimating

A lot of people in the Agile community these days are talking about estimation being wasteful and suggesting not estimating projects at all.

One radical view amongst all of this is to tell clients that as a vendor of a project

"I will try and deliver the best possible outcome based on resources I have at my disposal. I will give you frequent production quality software almost every other day for you to track how I am progressing. I will charge you per day or even per hour. Don't sign a huge contract with me but sign me up for a shorter span , say a month, review my software regularly to see what I have built and renew the funding for another month if you are happy. If you are not happy you can please kick me out. "

That might sound like the best thing to do or probably the most absurd depending on how your relationship is with your client. What people fail to articulate while suggesting this is that the entire premise is built on trust. And a whole lot of TRUST ! You tell a new client your are not giving an estimate and suggest the above dialogue you are bound to be frowned upon.

This view of stopping estimation is built upon the belief that "if we release software frequently, we do not need to worry about estimates". And this belief can be incorrect many a times.

For instance a new client who has never worked with you would want to compare bids from other similar vendors before choosing one. The issue becomes bigger in an outsourcing scenario where the main reason for choosing a vendor is cost. High level estimates on cost and time would definitely help the client make that assessment.

While releasing software frequently builds trust with the product owner(s), it might not build the same levels of trust with say the CXO group within the organisation. Maybe the CIO who is funding your project does not have enough time to attend your showcases or UAT sessions.

Asking a client to fund one month and fund the next month based on the product delivered might again sound good on paper but the reality is for a lot of big clients this means a lot of administrative overhead to manage. Choosing a vendor is a hard ask in itself and now being ready to switch vendors almost every month is compounding the problem.

Taking a hard stance by saying no to estimation will be highly immature in the above scenarios.But this does not mean all this is entirely impossible.  It is possible, but only after a period of time of working with the client. When a new project starts you have to bite the bullet and estimate. The important thing is to start moving towards a place where estimates become more and more irrelevant as you do more projects or more releases with the same client.

And this is where delivering value continuously will build trust with the client. Fostering people relationships at all levels with the client on top solid delivery will slowly nudge the client to start thinking about the product they are building rather than managing the vendor. This will then allow you as a vendor to focus way less on estimation and much more on building a great product.

Most "products" in the enterprise software world start as "projects" unfortunately. Projects are funded based on budget and the estimates are required. However changing this scenario is in the hands of the project team. It is the responsibility of the project team to slowly help the client focus on product building than project tracking. And that is only possible when there is enough trust built between a client and vendor, not always from day one !