Interview with Madhur Kathuria

Madhur Kathuria has coached nearly 300 teams for almost 75 clients across the US, Europe, South East Asia, Malaysia and Thailand. In this interview he talks about some of the cultural challenges for agile adoption. Read it here.

Interview with Elena Yatzeck

Elena was Chief Agilist for JP Morgan Chase Treasury Services and is now a VP of Corporate Compliance Tech. Find out how JP Morgan Chase reconciles agile with compliance and risk management demands. Read it here.

Tuesday, May 15, 2018

Is DevOps Agile?

So often I hear things like,
“The Agile methodology is different from SAFe”
“We are replacing our waterfall methodology with the Agile methodology”
Agile coaches will sigh at these kinds of statements, because they know that Agile is not a methodology. Those who are familiar with SAFe will sigh as well, knowing that SAFe is also not a methodology, but rather a framework for considering how to adjust one’s organization to accommodate Agile teams.

And then there is DevOps: is it a methodology? Is it an extension of Agile, or an evolution of Agile, or something different?

DevOps evolved independent of Agile because the Agile community drifted away from its technical roots, and therefore failed to keep up with technical advances such as cloud computing. Organizations needed to know how to scale Agile—not merely in terms of the number of teams, but also in terms of the complexity of systems that they could build. The scale of computing had greatly increased: companies today can have hundreds of millions or even billions of users accessing their websites: so-called “Internet scale” needed new approaches. The Agile movement began with eXtreme Programming (XP), which was highly centered on technical practices such as test-driven development (TDD) and continuous integration, but the Agile community failed to say how Internet-scale applications could be developed. Instead, the Agile community became mired in an excessive focus on process and team behavior, such as the way that teams plan their work, epitomized by Scrum.

And so DevOps arose apart from the Agile community. Initially it did not have a name: it was merely a set of solutions that organizations invented to solve real problems. Some of these solutions included (not a complete list),
    1.    Dynamic provisioning of virtual environments—aka “cloud computing”—to enable rapid integration testing and to enable new deployment approaches.
    2.    Containerization, in order to enable rapid turnaround in standing up new app instances on demand, and to provide isolation and repeatability for deployments.
    3.    Automated integration testing.
    4.    Large scale code repositories and integrating functionality across thousands of application components in real time.
    5.    Extremely thorough automated functional and non-functional testing, to enable continuous delivery and reliable and trustworthy continuous deployment, and ownership of deployment and support by dev teams.
    6.    Extensive real time monitoring of applications, to support a metrics based philosophy.
    7.    Increased knowledge of—and responsibility for—Web security by application teams.

All that time, the Agile community was busy debating whether organizations should first change their culture before trying to adopt Agile—and by “Agile” they meant Scrum, because Scrum took over the Agile community to a degree that Scrum almost became synonymous with Agile.

The rise of Scrum did great harm to the Agile movement, and indeed many have said that Agile is now broken as a result. (See my article here, and Dave Thomas's talk here.) For one thing, the availability of certification for Scrum resulted in a large number of people who had never programmed in their life obtaining an Agile certification, which they then used to get jobs as Agile coaches and Scrum Masters. Think about it: people with no programming experience telling programming teams how to do their job. It is no surprise that that did not work well. Today, there is a glut of Agile coaches, forcing down compensation; yet a very large percent of those coaches only know the Scrum process. I personally would prefer a coach who has years of experience on a programming team, as well as other leadership experience (of the servant leadership kind), perhaps even sales experience and P&L responsibility experience, because those kinds of experience make one sensitive to the real needs of a business. But the skills needed for a programming team coach is a lengthy topic in its own right.

The rise of Scrum occurred during the 2000s, with cloud computing coming onto the scene around the same time; but cloud computing was not really understood by most organizations until after 2010, and that is when the term DevOps came into being. By that time, the Agile community had finally figured out that Scrum cannot just be inserted into a large organization, but that other things also had to change to enable Scrum teams to work in that organization. The Agile community, by and large, did not understand large organizations, so its response was generally something like, “Don’t do Agile, be Agile”. However, that was not very helpful and it reflected the Agile community’s lack of knowledge of how to engage with organizations at the levels needed to make the necessary changes.

What happened next was that frameworks such as Less and SAFe came onto the scene. Less was generally supported by the Scrum community because it echoed that community’s ideology, which one could characterize as team-centric with a strong preference for self-organization and abhorrence of any kind of explicit authority. SAFe, in contrast, proposes a-lot of structure, and the the Scrum community has been very derisive of SAFe from the start. It continues to be to this day, to the extent that if someone wants to be a Scrum trainer, they are not allowed to also be a SAFe trainer—the Scrum Alliance will not certify someone who is known to be a SAFe trainer. How Agile is that? (If anyone wants to challenge my assertion, I have an actual case to back it up.)

Regardless of whether one prefers SAFe or Less or another framework, the important point is that the Agile community finally realized that its early dogma was too simplistic. It now has answers for how to “be” Agile, instead of simply saying that one should, as if it is the fault of the companies (the Agile coach’s customer) that they are not Agile—like they have a disease. Finally, the Agile community has useful answers to the question, What should we do?—other than “start doing Scrum”. It finally realized that to be something, you have to do something.

Meanwhile, the rise of DevOps took the Agile community by surprise, and now the Agile community has embraced DevOps as if DevOps is merely an extension or evolution of Agile, while the truth is that DevOps evolved completely independently out of the need to scale to Internet scale and deploy changes to Internet scale apps in rapid sequence. Fortunately, DevOps and Agile mesh together very well, because while the Agile community has chosen to focus mostly on process and behavior, DevOps practices are mostly around the technical questions of how to scale and rapidly deploy. Thus, DevOps has added a technical dimension back to the Agile viewpoint—a dimension which had been lost.

No comments:

Post a Comment