Books that helped me build better software delivery teams

I have learnt a lot by making mistakes on the job but there have been books that have influenced my thinking to a great extent. I will try to list the ones that were the most influential in changing my perspective towards managing projects and teams.

Classic General Management

• The Mythical Man Month - Essays on Software Engineering
• The Fifth Discipline - The Art and Practice of a Learning Organisation
• Peopleware - Productive Projects and Teams
• Waltzing with the Bears - Managing Risk on Software Projects
• What did you Say - Art of Giving and Receiving Feedback

Theory of Constraints

• The Goal
• Agile Management - Applying Theory of Constraints

Agile

• Extreme programming explained - Embrace change
• User Stories Applied
• Agile Retrospectives

Useful Link : Top 100 Agile books of 2011

Lean and Kanban

• Implementing Lean Software Development - From concept to cash
• Kanban
• Principles of Product Development Flow

Technical

• The Pragmatic Programmer - From Journeyman to Master
• Working Effectively with Legacy Code
• Refactoring
• Continuous Delivery

The Lean Startup

• The Lean Startup
• Business Model Canvas

There are many more great books out there but I would highly recommend the minimum reading list above for building teams to deliver great software.

Prefer partnering with craftsmen over contractors

If you are in an IT department in one of the bigger organisations working in the major verticals of banking, telecommunication, insurance, retail etc... you are probably looking at a portfolio of work which has major bumps in the curve when starting a big software initiative and a steady state when those initiatives move to a steady state of enhancements and bug fixes (a.k.a Business as Usual (BAU) ).

If you are someone responsible for delivering this portfolio of work you probably are tied under the typical IT budget constraints of a big organisation. Hiring highly skilled people quickly to deliver the bursts of projects that come along is going to be difficult. Given this scenario it is more than likely that your attention goes to the right end of the curve where you see the amount of work in the portfolio reducing within a number of years. This is when you think "Why do I need to hire more permanent staff, I can get things done by hiring contractors who can then leave when most of the work is done."

This might probably work if the contract staff form a very small percentage of your overall organisation. You might even be able to find some real good contractors who specialise in certain niche areas which might be temporarily beneficial to your company.

But the moment you start relying on contractors to do most of the work in the pipeline, it is the beginning of a big world of problems. Because here is what will most likely happen.

• You decide on hiring contractors to get some major pieces of work done.
• Since these contractors are temporary, you ensure that you have enough permanent staff to oversee the work the contractors are doing. You have most of the permanent staff in all the important decision making roles.
• The contractors come in and start working alongside your permanent staff. The good ones want to drive new initiatives within these projects but soon get frustrated by the red tape approval process created by your permanent staff.
• Your permanent staff start feeling insecure about their jobs once they see some contractors who are better at the job than themselves.
• You might try to hire these brilliant contractors which creates an ugly set of dynamics at work.
• The contract staff becomes frustrated and start working just the hours required for their contract. If you hired really cheap contractors then they are probably in this mode from day one.
• The permanent staff has issues with the quality of work done by these contractors and sometimes vice versa.
• You add rigorous shortlisting processes for contractors in a bid to weed some bad ones out. But you still end up with some bad contractors because of budget constraints.
• Your project ultimately might go live but chances are that the quality of the architecture, codebase, user experience , performance is worse than ever. Your business stakeholders are unhappy.
• The contractors leave and you inherit a horrible quality system.
• Your permanent staff feels terrible maintaing this poor quality system. They either suggest a complete rewrite of the system again or suggest outsourcing the maintenance of the system to contractors.
• Another project comes along and you are probably still choosing contracting firms in the never ending grind of work.

So is there a better solution ? Let us look at the assumptions again

• Most systems you are building are of strategic importance to your business.
• There is more work in the pipeline than the permanent staff can chew and to be delivered in an aggressive timeframe which does not allow you to hire more permanent staff.
• The inflow of work might reduce after most IT systems are up and running in a few years.

and of course the more important long term goals that

• You need to be build a high quality system or platform which scales well to serve the business needs.
• You need to build an IT department with skilled technical people who are able to deliver and maintain the systems required for the business

So it is in your best interest as someone tasked out for delivery to build good software and at the same time set out to build an organisation which can do the same repeatedly in the future.

This is best done by hiring good people in first place, but if for various reasons you cannot, then at least try to partner with people who are passionate about the craft of building software than people who worry about billing the 8 hours of work.

To scale, try to find companies with whom you can partner with for their delivery services knowing how culturally passionate the partner company is about delivering the work for you. (And believe me , there are many good consulting companies out there which can help you out).

And by partnering, you engage with them at levels where they have a decent voice in decisions made on how to deliver the project. There is transparency and a drive towards building a win win collaborative culture within the team. You treat them as strategic partners rather than augmented resources in your delivery organisation .

It takes time to find such companies to partner with. You need to validate their delivery capability and culture fit over a period of time. But once you have done that, try and build a lasting relationship with them because in reality you will always have a backlog of work given the dynamic nature of your business.

After all, you are building long term assets for your company and it can best be done by partnering with people who care about it the same way you do. So invest in finding a good software delivery partner that can work for you instead of risking long term failure by using low cost/quality contractors.

When Agile meets yearly IT budget

There is a well-known problem when Agile projects meet a client’s yearly IT budgets. The problem with the yearly budget is that

1. It is largely incorrect as it has been planned with minimal information

Most big organizations have a budget review only once a year where a CIO and his team have to decide what their future looks like for the year. This is the time when everyone looks into their magic crystal ball to estimate the amount of funding required for their projects in the financial year.

These numbers are largely incorrect purely due to the scale at which they are done and with the limited amount of information available upfront.

2. Fosters project thinking and deliver under budget than building the right product.

Once a ballpark yearly budget figure is reached after minimal planning by project teams, the teams are encouraged to plan in more detail to secure their exact funding required.

Project teams then start building their detailed product backlog of stories with estimates and break it down into releases through the major part of the year to come up with their funding numbers which are then secured.

This not only results in a lot of upfront planning even in Agile, but also focuses the team on delivering that huge backlog of stories under the project budget, than focusing on building the right product fit for the market. Needless to say that these big backlog of stories often get outdated within the first few iterations and so does the estimates and hence the budget plan.

So is there an alternative?

The good news is that a lot of companies are beginning to recognize this problem. And these are companies who view their products as “Strategic IT” investments and not an operational overhead. The solution requires changes in 2 key areas, budgeting and project planning and tracking.

Rolling wave or Quarterly budgeting

A year is clearly too long to forecast and budget for upfront. One option is to budget for the first couple of iterations and then budget the third and fourth just before the completion of the second iteration based on real data. This ensures that at any point in time you have budget for the next couple of iterations.

If a couple of iterations are too less for long-term visibility, the other option is to do it quarterly.

The advantages of this approach are

1. More informed hence accurate budgeting based on actual data on the ground.
2. Encourages people who own the high level budget to engage with the team on the ground at least once every quarter.

Track projects to measurable goals and not to a product backlog.

Not having to forecast a yearlong budget is a boon to an Agile team. The team can then focus on building just enough of the backlog for their product to be able to validate it with end users and then pivot or preserve as necessary.

This is a mindset shift to the traditional Agile/Scrum technique of doing discovery workshops and then building a backlog of user story phrases that are then elaborated every sprint.

Instead, the team needs to come up with clear and measurable goals on how they measure the success of the product. This could be things like increasing user uptake of the product by releasing some new features, improved customer service, launching a new brand / service etc.… As some of these goals might be long term, it is important to break them further down to a level that can be measured every month or a quarter. An example would be to validate an idea for a web page with 5 users before building 3 similar web pages in the subsequent iterations and validating with 50 users.

The owners of the budget in this case can decide based on the progress towards goals of the first few iterations to fund / invest in the next few iterations. This also gives them on option to pivot or preserve based on what they see built so far.

To summarise, the advantages of this approach are

1. Less of time spend in estimation of effort and more time thinking how to build something valuable
2. Keeps stakeholders who make budgetary decisions more engaged in the product.
3. Makes product owner more responsible to think and show results building the right product than forcing the team do deliver a backlog

References

The Budgeting Black Hole - Johanna Rothman

Annual Budgeting and Agile IT - Ross Petit

Lean Startup – Eric Ries

Implementing Beyond Budgeting – Bjarte Bogsnes

The problem with Continuous Delivery & DevOps as an IT investment

In Thoughtworks we have a lot of experience in implementing Continuous Delivery , and we have done it project after project under some name or the other. In fact we did it long before Jez even wrote his book on the subject. (Jez talks about one of those projects here). I remember roles which we used to call Build Masters/Engineers, even Build Monkeys and teams such as Build and Environments team which practiced the same principle. These teams or roles used to be specialised and sat outside the group of people who delivered functional user stories.

Today we call the people DevOps and the practice Continuous Delivery, but largely we have the same problem as 10 years ago. DevOps is seen as a separate team and Continuous Delivery is seen as an IT initiative. In fact organisations have gone a step further in this direction to appoint IT heads for continuous delivery who are in charge of such teams.

The reason Continuous Delivery becomes an IT initiative

1. Business sees it as an IT investment

In most big organisations business teams are still keep a safe distance from the gory IT details and more so from IT Operations and Support teams. (Business teams have more to worry about in their lines of business). In their view Continuous Delivery is an IT Operations issue solely as it will minimize the operation spend for IT. They see the key driver being within IT and the impact only on the IT operational cost and hence believe that Continuous Delivery should largely be an IT investment and if an organisation wants to do it, the IT budget should cover the cost and not the business.

2. The practice is seen as a mighty legacy systems/operations overhaul

In most large (> 1000 people ) organisations such as banks and insurance firms there are a whole lot of legacy infrastructure lying around. These are systems on which no one has ever attempted writing a single automated test. Deployment is a humongous tight coupled multi-step process which no one has ever imagined being done with a batch job.

In such places an IT organisation has to start somewhere and that cannot happen without a focused initiative. This results in a new department whose sole job is to focus on bringing in the continuous delivery practices into the organisation.

Problem with a separate DevOps team with an IT budget

The problem with a DevOps team funded by an IT budget is that it results in another glorified Operations/Support team, only this time balanced with application developers and sys admins. Being centralised to project teams, this team ends up doing any work classified as "Infrastructure" such as provisioning and configuring virtual machines, optimising and maintaining the build environments, production server monitoring etc...

Product owners, on the other hand, push their teams to deliver more functional stories and encourage them to delegate any "technical/infrastructure" work to the DevOps team. This results in product owners distancing themselves from the benefits of DevOps which is contrary to the whole principle of Continuous Delivery, which is to demonstrate rapid responsiveness to change and hence create a strategic impact for the business.

Moreover some DevOps teams work with a goal to make themselves redundant and merge themselves with functional product lines. This goal often remains an aspirational one with their ever growing backlog of work.On the contrary DevOps teams end up creating their own silo with specialised skills which developers on project teams struggle to keep up with.

There are enough reasons for business to cheer about Continuous Delivery in the longer term. An investment from a business unit will only result in business teams working closely with IT, which is at the heart of Agile and Continuous Delivery. Business teams will be able to find new ways of creating a bigger strategic impact and have an edge over their competition. Hence for the right alignment within the organisation, funding for Continuous Delivery should come from a Business budget.

Why you should care about experience design and continuous delivery as a project manager ?

I had written earlier about how a good project manager in a mature Agile team needs to go beyond managing scope, time and budget to be really effective in his/her role. The key is to understand the product and the tools and techniques used by the team to build it.

But there is more a project manager should do, which is, to provide the right direction to the team. The focus shifts from "building the thing right" (i.e delivering within scope , time and budget) to "building the right thing" (a product that fits the purpose the business and its users).

Building a delivery engine that can react quickly to market feedback and adapt needs focus in a few different areas than traditional project management. And this is where project managers should have a good understanding of experience design and continuous delivery.

From a continuous delivery perspective a PM should have a good understanding of

* The level of automated testing required to certify almost production quality builds in no time.

* The value of using a production like environment for testing and feedback.

* How automated deployments can help speed up time required to deploy to multiple environments and rollback quickly if the need arises.

* The need for real time monitoring of production systems for the team to quickly respond and fix production issues.

* How one can benefit from feature toggles v/s branching.

* The advantage of releasing quickly and minimising work in progress to shorten the feedback loop.

The main advantage of continuous delivery is not so much in reduction of deployment and maintenance costs. Instead it is in the capability it provides to validate and adapt the product being buit quickly based on end user testing and research.

It is important , as a PM to know

* If we are doing enough user testing to validate what we are building at regular intervals ?

* Are we doing enough user research to provide critical user context data in the form of personas etc...?

* Do we have good analytics capability built in the product to profile usage patterns which can help us make informed decisions about the product?

An insight into some of these aspects can really help a project manager steer the team towards building the right product which is way more valuable than someone who ensures the team is tracking to an outdated plan.