Prediction #10: Agile rollouts increasingly driven from executive office, developer backlash follows

by Mik Kersten, January 26th, 2011

Agile development is at a crossroads. Practices hardened around small teams of motivated enthusiasts are now being deployed at scale in the enterprise. Average professional software developers, far from the Scrum rock stars or Kanban aficionados, may remain inspired by their Agile training for a couple of months. Then the next release crunch comes, and too often the Agile process is perceived as yet another source of overhead that keeps the developer from coding. Agile practices which have not yet become engrained, even though they are recognized as beneficial, start to fall by the wayside. User stories are not broken down into tasks until the end of a sprint, release frequency drops, the Agile project tracking tool is only kept up-to-date at the insistence of the project manager masquerading as a ScrumMaster. The software development process is once again driven by managerial needs rather than those of the self-organizing team.

The unintended consequence is that the Agile process has degraded into a project documentation methodology. While managers who architected the large-scale rollout scramble to adopt even more cutting edge Agile and Lean practices to address these problems, upper management becomes concerned that they now have even less visibility into the development process than they did with waterfall. Arguments are made that teams’ velocity has increased, but the benefit is outweighed by the overall loss in predictability and increasing developer discontent. At this point, the expected ROI of Agile is looking more like a realized loss.

The frequency of stories like the above is increasing with the growing number of early adopters of Agile at scale moving beyond the initial honeymoon phase. For organizations scaling their Agile deployments into the thousands, it is not be feasible to train and motivate thousands of employees to do Agile as it is “meant to be done”. But complaining that developers are not doing Agile right is akin to telling iPhone 4 users that they’re “holding it wrong”. To scale to large enterprise, a new work process has to be encapsulated with tool support that embraces the individual contributors doing the actual work. To date, Agile vendors have done a very good job providing project managers and product owners with planning and tracking tools. However, the primary creators of project deliverables are software developers, and they have consistently been an afterthought in most Agile solutions and deployments and are in the process of being disenfranchised.

The crux of the problem stems from the inherent need for large scale adoptions of Agile to be mandated by management. Consider the previous technologies that followed this trend. The Unified Modelling Language (UML) seemed like a great way of getting business level requirements discussed as part of software design. Heavyweight Application Lifecycle Management (ALM) tools from a decade ago promised to transform the software delivery process. Both saw a backlash from developers who knew all along that they reaped more benefit more from lightweight tools that embrace coding and collaboration.

This decade-old backlash against heavyweight and management-driven ALM tool deployments was part of the inspiration for the Agile Manifesto’s call for “Individuals and interactions over processes and tools”. If we keep heading in the current direction of ignoring the need for tools that bring developers’ programming activity into the process and that provide them with a concrete and visible benefit to their day-to-day coding, we will witness the same kinds of failures with Agile that we saw with ALM tools. We have slid far enough down this slope that in 2011 we will start to witness both major hiccups and outright failures among early adoptions of Agile at scale.

To address this necessary move of Agile from early adopters to enterprise pragmatists, tools that focus on individual developers needs are the critical “last mile” of the Agile deployment. We need look no further than the Toyota Production System (TPS), the precursor of the Lean manufacturing trend which did much to inspire Agile development, for guidance on the role that software production tools need to play in the process. The TPS consists of three interlocked elements: the philosophical underpinnings that define what we deliver to the customer, the managerial culture that defines how teams collaborate, and the technical tools that get things done. Developer tools are as primary to Agile software production as assembly line automation is to cranking out Toyota Prius automobiles. The existing focus on the ALM and above layer needs to be extended down to the Integrated Development Environment (IDE).

Coding, deployment, source control, and continuous integration tools must be integrated as a key pillar of the Agile development process. The knowledge formed around development activities needs to be captured and linked within the plan and product backlog. User stories need to be automatically connected to commits, and the developer’s workspace focused away from the system as-a-whole and onto the task-at-hand. Developers need to be given powerful and efficient tooling which gives them a window into their part of the Agile plan and connects their workspaces to the delivery process as a whole. To avoid the backlash of a thousand developers complaining that the changes imposed by an Agile rollout are getting in their way, we need a new focus on the final and most critical gear in the Agile deployment—the developer’s workbench.

11 Responses to “Prediction #10: Agile rollouts increasingly driven from executive office, developer backlash follows”

  1. Tim Says:

    I’d like to see a roadmap for implementing this solution (tasktop) for developers. All I see are demos of the working product – how do I get there?

  2. Mik Kersten Says:

    Tim, thanks for calling me out on whining about the problem without pointing to a clear solution. I have been so focused on the solution of late that I wanted to step back and kick off some discussion about the underlying problem. Here are some quick thoughts on how we often see the roadmap unfolding, with they key being getting developers to activate and work directly on tasks.

    * Get developers working against their Agile tasks directly in the IDE. This means deploying developer integrations for Agile (eg, Mylyn/Tasktop, Microsoft VSTS). Emphasize not the benefit to planning, but that of making their coding activities easier, eg, activating the user story or defect you’re working on makes coding easier by focusing your workspace and giving you instant interruption recovery.

    * Have all developer collaboration happen via the IDE integration and Agile/ALM issue tracking or change management tool. Get out of email and IM for delivery-related discussion. The plan and backlog need to capture all the knowledge related to the product, and this will help developers get benefit from working and collaborating via tasks/stories/defects. Our vision here is that Tasktop+IDE+Agile/ALM servers become the developer’s Outlook+Exchange server

    * Connect this developer activity with your Agile ALM and PPM tools. For example, having tasks be an explicit part of the development process means to-the-minute actuals rolling up from user story and higher, which transforms estimation from guessing game to reality. Doing this with tools that give developers more control, not less, means avoiding big brother concerns of over-monitoring activity.

    I realize I’m using very broad strokes here. The points above and the problem addressed in the blog post are some of the foundations of the Tasktop product releases for the first half of 2011. Let me know if this is along the lines of the sort of roadmap that you’re looking for, as I think this could warrant a separate post outside of this series.

  3. Mike Says:

    How is this different from Jazz and IBM Rational Team Concert?

  4. Tony Baer Says:

    Regrettably, a lot of the popularity of agile comes for the wrong reasons:
    (1) developers who misconstrue “Individuals and interactions over processes and tools” and “Working software over comprehensive documentation” to mean, get software out and don’t worry about process.
    (2) Enterprises whose agile initiatives are more buzzword- than process-compliant.

  5. Tony Baer Says:

    Given the grassroots origins of agile, your headline is ironic and all-too-often sadly true.

    Regrettably, a lot of the popularity of agile comes for the wrong reasons. For enterprises, agile initiatives are all too often buzzword- rather than process-compliant.

    For developers, it’s often misinterpretation of what the Agile manifesto really means. “Individuals and interactions over processes and tools” and “Working software over comprehensive documentation” are misconstrued to mean get software out and don’t worry about process.

  6. Mik Kersten Says:

    > How is this different from Jazz and IBM Rational Team Concert?

    Mike, assuming you’re asking about the related solutoins and not the post, Jazz/RTC is a unified ALM stack providing CM, SCM and CI developer integrations for Eclipse. What we do is provide developer integrations for the vast majority of the leading ALM vendors in order to support those using best-of-breed ALM stacks. That including RTC, eg, if you want to use RTC for your planning and issue tracking, but use HP QC for defect/quality management, Git for SCM and Hudson for CI. In addition to developer integrations, we provide the task-focused interface, which is relevant for developers on a pure RTC stack.

  7. Mik Kersten Says:

    Tony,

    Yes, the irony here is thick enough to cut with a knife. When I was at PARC, and our AOP stuff was getting popular, a senior manager familiar with technology lifecycles cautioned us saying “Watch out, when a new technology starts smelling like money a whole new breed of dogs moves in”. There is clear value in Agile, but as you point out, the growing popularity of Agile means that buzzword-compliance is indeed driving deployments, and that kind of thinking will get in the way of the goals of Agile in the first place. And yes, the fact that the less passionate developers are misconstruing Agile as a way to get out of process is just going to fuel the fire. At least Agile has its grassroots in developer culture, unlike other new approaches that saw a similar failure mode, such as UML. So there is hope that we sort this out before too many deployments blow up.

  8. Tony Baer Says:

    Exhibit A: I know of one large enterprise whose “sprint trackers” are nothing more than detailed timesheets that add more overhead to daily work.

  9. Tasktop Blog » Blog Archive » Newsletter: New SmartBear CodeCollaborator Connector, Visual Studio public beta now available Says:

    [...] Prediction #10: Agile adoption continues its managerial rise, developers get annoyed and cause a backlash [...]

  10. Tasktop Blog » Blog Archive » Tasktop 2.0 released with Eclipse Mylyn 3.5, brings sanity to your heterogeneous ALM stack Says:

    [...] content, customizations and workflows encapsulate product history. Agile tools have moved in, but have not been adequately connected to developer workflow or to enterprise ALM tools and processes. Lightweight open source ALM tools are becoming favoured [...]

  11. James Sinur Says:

    Business executives want a visualization of the end process/application. I would suggest that modern business process management technologies agressively support agile while giving the business folks a visualization of the end results early. If I was a CIO, I would try and cross pollinate agile developers with BPM efforts to create a win win. Just my two cents.

    Regards,

    Jim

Leave a Reply