Interview with Madhur Kathuria

Madhur Kathuria has coached nearly 300 teams for almost 75 clients across the US, Europe, South East Asia, Malaysia and Thailand. In this interview he talks about some of the cultural challenges for agile adoption. Read it here.

Interview with Elena Yatzeck

Elena was Chief Agilist for JP Morgan Chase Treasury Services and is now a VP of Corporate Compliance Tech. Find out how JP Morgan Chase reconciles agile with compliance and risk management demands. Read it here.

Tuesday, November 4, 2014

Key Transformation Decision: Adopt an Out-Of-the-Box Approach, Or Craft a Unique Approach?

Organizations like quick answers. If you need to get into a new market, buy a company that is in that market. If you want to make your IT function more agile, pick an agile framework and “install” it. Such frameworks include Scrum and Scaled Agile Framework (SAFe). Both of these provide certification regimes, which organizations like because it turns the skill set into a commodity that can be purchased – i.e., “installed” in a repeatable manner.

Many people in the Scrum community will object at this point: they will claim that Scrum should not be conflated with SAFe. Indeed, the two are descriptions of agile at very different levels. However, each one is a prescriptive model for how to implement agile, whereas the Agile Manifesto is a set of values and principles that one can implement in an infinite variety of ways. So yes, both Scrum and SAFe are “out of the box” frameworks.

That is not a bad thing. Scrum is a great approach for team level agile. SAFe has some really great ideas in it for how to support agile projects in a large organization. These frameworks serve as excellent models for thinking about how to do agile, and in fact it might sometimes be possible to implement them exactly as they are defined. But that is not always the case.

The real question is, what should an organization do to get started with agile? Should it try to adopt a framework? Or should it think through the Agile Manifesto – and other spheres of thought, including Lean, org change, and so on – and define its own path? This is an important strategic decision, and it is important to get such key decisions right early on. (See the article It Is Important To Get Key Choices Right.)

The argument for using an out-of-the-box framework is that it has all been thought through and tried and tested. But that does not mean that it can be applied successfully in every organization; nor does it mean that the path to implementing the framework will be easy. In fact, these frameworks are really future state models, but the hardest part is figuring out how to get there.

One problem with using a framework as-defined is that doing so is inherently non-agile. From that point on, the team looks to the “expert” for guidance on how to do their work.

One problem with using a framework as-defined is that doing so is inherently non-agile. If you force a framework on people, and give them an expert in that framework, you have essentially taken their process away from them. From that point on, they look to the “expert” for guidance on how to do their work. This is completely contrary to the agile value of “Individuals and interactions over processes and tools”, and also contradicts the agile principle of “Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done” – i.e., trusting people to figure out how they should do their work. Giving someone a framework and saying “do that” is telling them how to do their work, and is contrary to agile values and principles.

An alternative is to bring in an expert in agile – ideally someone who is knowledgeable of multiple frameworks and multiple spheres of agile thought – and have that expert discuss the agile values and principles with the team in the context of how the team is currently doing its work. Then, work with the team to apply agile values and principles to come up with improvements to how they work. The frameworks can serve as sets of ideas for how to improve things. These improvements can be incremental or radical, depending on the team’s appetite for change – but the team is then in control of the change. The team owns their own process, and does not need to ask the “expert” for permission to try something new.

Let me be clear here that I am not against Scrum or SAFe, or any prescriptive process – in fact, I think they are really important agile models. It is just that I think that teams should decide on their own process, and not have someone tell them that they should use Scrum or SAFe or use these in a certain way. The team needs to be able to understand every change they make to their own process, and have their own performance improvement goals for it. If you give them an entire canned process, and tell the team that all of the practices work together, then the team does not know what to expect: they have to trust the expert and accept on faith that the new process will work. They don’t truly understand it – only the “expert” does. The team has to wait until they have also become experts before they can appreciate the new process. That is not an agile way to do things. That is “big process up front”. It is command-and-control. It is all at once, rather than incremental. It is about “processes and tools” rather than individuals and interactions. It is exactly what we tell people not to do, when we tell them to “be agile” rather than “do agile”.

The Alternative: Take an Issue-Focused Approach

Rather than starting out by telling a team that they need to adopt a framework that they do not yet understand, talk to the team about their challenges. Take an issue-focused approach. Does testing result in lots of rework? Are there bottlenecks, possibly due to certain people having certain tasks that others have to wait for? Is there contention for testing environments? Is there a requirements or design document, but no one to ask about it? And do the requirements or design sometimes need to change, but changes require an onerous change control or review process? Are there third parties who need to perform tasks such as independent testing or security review, and those tasks take a long time and can potentially hold everything up, or result in lots of unanticipated rework?

These are all tangible issues that agile – including frameworks like Scrum, eXtreme Programming (XP), SAFe, Lean, and other agile models and schools of thought – can help with. For example, on the first issue about rework after tests are run, the team can discuss how it might implement a continuous integration (CI) cycle, so that tests are run locally by developers and that way there are no surprises when tests are run in a test environment. With respect to bottlenecks, the team can discuss how to enable more people to work on the items that are bottlenecks – possibly by training certain people using pairing – or if that is not feasible, perhaps the bottleneck work can be started earlier so that it is staggered with respect to dependent work, creating a kind of pipeline or Kanban process. In fact, tangible agile solutions can be found for each of the team’s issues. The best part is, the team is not thrown into an entirely new process overnight, but instead they are inventing their own solutions for their own problems – and moving toward agile in the process.

This approach will improve productivity from the very beginning.

This approach will improve productivity from the very beginning. There will be only a minimal Satir change curve dip, because the changes are incremental and not disruptive. The team will also have higher morale, because the innovations are their own, and they are in full control of their own process. And their old process has not been thrown out the window and derided as “bad” and “waterfall” – synonyms in the agile community.

There is a catch with this approach as well: It will only work if there is authority behind the changes that are needed. Within the team, the team has to be empowered to make its own decisions about how it works, without interference from project management or external entities. Also, the team has to be motivated to improve things, and not stuck in its habits. Some people resist change, even if it might improve things. A team that is composed of people who resist change will not want to devise improvements to its own processes.

If the team resists change, then the agile coach needs to have authority to impose a mandate on the team, such as “We need to improve velocity by 20% over the next four weeks”, or “We need to reduce testing time by 50% over the next two months”. The coach’s mandate can derive from a CIO mandate giving the coaches authority. The CIO mandate can even be specific, by mandating certain agile practices, but that is not a very agile approach – it is prescriptive, just like mandating a framework – yet it can work if the coach is around sufficiently to guide the team in those new practices. In contrast, if the team devises its own way to meet goals such as decreasing test time, they will need less guidance.

Even if the team is motivated to improve its practices, they might still have to comply with externally imposed non-agile practices such as gate reviews and manual testing by an external testing group. In order to negotiate how those constraints can be relaxed, the agile coach needs authority to be able to make “deals” on behalf of the team. In that capacity, the agile coach is acting more like a project manager, even though when dealing with the team the coach acts as a coach. Leadership requires applying the right methods depending on the situation.

The same lessons about imposing a framework apply at an organization level. In fact, at an organization level, it is even more important to allow people to define their own business functions. To do that, people have to first learn about agile. They have to visualize what their function would look like under agile – or even if their function would still exist. Then, they have to make concrete plans for changing their function, with the assumption that change will occur in stages – it will be unusual to get it right from the outset. All that takes time. Pushing a pre-defined set of processes on an organization will likely create a-lot of frustration because the “train is moving” and managers are still on the hook for delivery dates and meeting their various commitments, as well as complying with organization policy that is still in place. Creating entirely new processes and systems takes time, and people need to learn how to operate those new systems – and new ways of thinking to boot – and that takes even more time. This is why a framework – such as SAFe – is a great place to start for ideas, but should not be implemented “out of the box”.

The Big Framework Up Front Can Work

Transitioning a team overnight to a prescriptive agile process can work. I have seen it work – but not often. One case in which it worked well was a project at a company called BCE Emergis during the early 2000s. The project had 80 people (about 50 developers using RUP). The company decided to rapidly adopt eXtreme Programming (XP) and brought in XP coaches to help. We carved out four development teams. I was the lead of one of those teams. The rapid transition to XP worked, because each coach was highly experienced with XP, and there was at least one coach per team. Thus, the coaches were present all the time, and participated in discussions – both planned and ad-hoc.

This is the “Shuhari” model, or apprentice/craftsman approach: an expert (the craftsman, or master) trains an apprentice. The apprentice is expected to follow the direction of the craftsman to the letter until the apprentice learns the craft well enough that he or she can do it on his or her own. Eventually, the apprentice becomes expert enough to make his or her own judgments – even to the point of violating the practices that he or she was taught, if the situation warrants it – in other words, the apprentice has become a craftsman, and understands the reasons for the rules well enough to bend or break the rules when it makes sense to do so. For the apprentice/craftsman approach to be an effective instruction/learning strategy, a craftsman is needed who can spend a great deal of time with the apprentice. This approach does not work if the craftsman or master is seldom available.

Most organizations that adopt agile do not do it in a Shuhari or apprentice/craftsman way: in most cases that I have seen, each coach has a portfolio of projects that they coach – typically from three to ten projects/teams. And for coaches, I am not including recently trained Scrum Masters, who just obtained their two day Scrum certification and have never worked on an agile project before. Those people are beginners every bit as much as the rest of the team is.

Three to ten teams per coach is too many: the right ratio is 1:1. If it is not 1:1, then the team is essentially expected to use an entirely new process that they do not understand, and there is seldom anyone around to guide them. They end up trusting the process and not thinking for themselves, but no one is around to tell them what to do. This can have a very bad outcome, including testing that gets overlooked, technical work that goes undone, and important collaboration that fails to occur, and – worst of all – the organization sees poor results from agile adoption and loses confidence in it.

Conclusion

I really like the expression “fish or cut bait”. If you are going to do something a certain way, really do it. If you are going to adopt a framework, bring in the resources to do it for real – not half way. If you cannot afford those resources, use a different strategy that you can afford and do it right. If you have a few coaches and many more teams, then you cannot effectively use the Shuhari or apprentice/craftsman approach. Instead, leverage the brains on the teams to adopt agile: use the coaches to help the teams to talk through issues, giving them an agile perspective, but don’t expect the teams to adopt an entirely new way of doing things. Empower them to adjust their own processes – which they understand – gradually, learning as they do it. That is more effective if the coach is not always around, in which case the team needs to be able to think for itself, on its own, even as they are learning agile. That is being agile.

Then again, these are complicated issues, and to say that a framework should or should not be imposed in a given type of situation is itself prescriptive. More often, the right answer is a combination of approaches: perhaps some prescription is called for, to push things along, while allowing people to innovate with their existing processes as a starting point. Judgment on the right balance is called for. Advice on that judgment is the value added by agile coaches, at both the team level and the enterprise level, and managers need to apply their experience with respect to organization change in order to find the approach that is best for their organization’s culture and circumstances.

Contributors (alphabetically):
Cliff Berg
Sekhar Burra
Sunil Mundra

No comments:

Post a Comment