Intro to Agile (Part 2)

In the last post i went over the first half of my presentation on agile software development. We talked about the purpose of agile and how, at a fundamental level, it's all about the way you think and not the things you do. That post was WAY more important than this one so if you're starting from scratch in agile I highly recommend reading that first.

Today I'm going to cover the various tools, practices and patterns people apply to try and be an effective agile team.

Agile Project Management

There are three overarching project management techniques that are applied to agile projects. They differ in their implementation but all focus on the same concept of continuous feedback and close collaboration. They are:

1) eXtreme Programming (or XP) - This was the first technique I heard of and in some ways it is the "strictest" of the agile methodologies. In focuses on short development iterations (or sprints), a series of planning meetings throughout the process and sound development practices like test driven development and paired programming.

2) Scrum - Scrum is an incredibly popular technique among enterprise agile implementations. It is very similar to XP in day to day implementation but also covers the scenario where a product is being developed by a large team and how you would go about managing that. Specifically scrum speaks about breaking large teams into smaller scrums then developing some sort of process (called a scrum-up) where the smaller teams report to a higher organization to share information. Development practices are more left to the individual implementations to decide but TDD is generally encouraged in all of the agile implementations.

3) Kanban - I perceive Kanban to be one of the newest options but it is getting a lot of traction. It emphasizes a continuous flow of new functionality with intermittent triggered events that might bring about planning meetings or retrospectives. It is heavily focused on pulling story cards through the process represented by a Kanban board the team views the project status from.

Which of these options you choose is up to you and your business. For the type of project work we do at my company Kanban seems like the best fit, but I have worked for in house IT departments where Scrum or XP was perfect. As with most things in agile the key is being flexible.

Storycarding

One of the common practices that occurs in any of the methodologies listed above is storycarding. Put simply storycarding is the act of breaking up an applications requirements in to small chunks and framing the description of that requirement from the perspective of the intended end user. Put even more simply its turning your product into a series of small stories.

With storycarding we have a few rules:

1. Each card should tell a story about how a user will interact with the system


2. A card should have some business value


3. Each story should have as little complexity as possible


4. Stories should consist of as little detail as possible

Numbers 1 and 2 above leave us with a problem. If every story has to be about the user and they all have to have concrete BUSINESS value how do we account for tasks that are purely IT driven, like "Setup infrastructure" or "Upgrade to .NET 4.0?" As with most things in our profession the answer is "it depends." Purists would say that those tasks need to be taken inline with some other business focused task that drives them. For example your very first story could be used as a catalyst for "setup infrastructure" because without that infrastructure that first story can never be released and thus is never complete. Some more practical teams opt to put in technical story cards and negotiate with the business time and prioritization of those tasks. Ultimately this is up to the team.

Numbers 3 and 4 go hand in hand as well. We want out stories to have little complexity so that they are each to test, complete and assign. We don't want to have stories that take weeks to finish and we don't want a lot of overhead in order to get started on one.

The reason why we favor less detail in a story is to encourage conversation and collaboration. The more we put in writing in advance of the work beginning the more likely it is that we will need to change the details later on, most likely after the card has already been completed as written. Instead a storycard is treated as an "invitation to have a conversation" with our business partner.

Managing Work-flow

So knowing how agile teams usually generate application requirements is one thing, but what do you do with all those story cards?

This is an area where the different agile approaches differ. XP and Scrum are generally in line so we'll talk about their approach first.

XP and Scrum focus on working in short sprints (also called iterations.) In this system work is typically done in 2 week spurts of activity (although the actual amount of time varies by team.) Each sprint follows a cycle that consists of a sprint planning meeting, a period of work and collaboration with the business followed by a retrospective, then the whole process starts again.

Sprint Planning Meeting

A sprint planning meeting is a time period where the team evaluates how they are doing so far (retrospective) and plans out storycards for the next sprint. Typically the business will assign priorities to cards and the cards will be scheduled for the sprint in priority order. Exceptions to the priority order rule occur when one card is dependent on another or the business, having seen the progress the team is making, decided to re-prioritize a card during the SPM.

The actual process associated with an SPM varies by team but in general it begins with a retrospective on the previous sprint. It then involves someone reviewing what tasks in the previous sprint were finished and which ones need to be rescheduled, potentially into the upcoming sprint. Some teams then opt to do a live demo of the currently in development system. Finally the business works with the team leadership to prioritize the remaining cards and schedule them into the sprint. XP uses the concept of "yesterday's weather" when determining how many stories make it into a sprint. Basically this means that however much work was accomplished in the previous sprint if the amount of work you anticipate for the next sprint. Any remaining stories are placed in a buffer to be picked up in priority order if all of the scheduled tasks are completed.

Release Planning Meetings

Scrum and XP also have a concept of a release planning meeting or RPM. An RPM occurs in advance of the kickoff of the project as a whole and is used to plan out how the major areas of functionality will occur throughout the course of the project. The estimations and schedules that come out of an RPM are really fuzzy but they give the team an roadmap of sorts as they progress throughout the project. Some teams will create storycards in advance for the first few sprints and pre-schedule those cards in the RPM. This process works well enough so long as the result is seen as flexible. Those schedules will change as you learn more.

The Kanban Approach

Kanban takes a different approach from Scrum and XP. Instead of having prescheduled meetings and working in short sprints Kanban focuses on a continuous pull of new functionality into the system. Reviews, retrospectives, demos and planning sessions still occur but they are triggered instead of scheduled.

A storycard in Kanban begins prioritized and ready to be worked on. Team members pull cards from the ready bucket into the development bucket, as they are finished they move to QA, staging and deployment (or any number of other buckets depending on the team.) New cards are planned when the ready bucket gets low. What "low" means is up to the team. For one team they might decide that when there are only 4 stories pending they need to sit down with the business and plan out more stories. For another team that number might be 20. The same applies for demos or retrospectives. One team I saw triggered a formal retrospective when enough post-it notes were left by team members on a "retrospective" board. So as team members saw good, bad  and ugly things happen they would leave notes on the board (anonymously if they choose) and when a certain number were there they would all sit down and review them.

The key thing to note here is that Kanban really doesn't "stop." There is no break to plan an IPM and then restart work. Team members keep building for as long as there is work to be done.

The Daily Standup

The final meeting that seems to be consistent across Scrum, XP and Kanban is a daily standup. A standup meeting is a short meeting held daily where all of the development team members and sometimes testing and business will get together to go over the current state of the project. In general everyone in the team should be able to answer three questions.

1. What did I do yesterday?


2. What am I doing today?


3. Do I have any roadblocks?

Should a team member encounter any problems the team should match people up to overcome those obstacles and move on. In general an ideal stand up is no more than 15-20 mins long.

Sprint 0

There's a lot of debate as to what an Agile team can do in advance of a project. The notion of pre-planning and pre-work has come to be referred to as "sprint 0." Generally the point of Agile is to adapt as time goes on and to not make decisions well in advance of when they need to be made. The notion of a period of pre-planning and decision making seems to fly against the goals of an agile team, but practically there will always be a number of tasks that must occur before the project can get rolling.

The rule of thumb for Sprint 0 is to not do any work or make any commitment to a course of action that could change based on the businesses requirements. If a task is aligned with the requirements of the project it should be accounted for in a storycard and prioritized into the project.

There are a number of activities that do make sense though. You can identify team members, stakeholders and set expectations for the project. In most situations you can decide on an overarching technology, for example if you tend to be a .NET shop you might choose between webforms or ASP.NET MVC. Finally you might identify the very large functional areas you want to hit on for this project, i.e. you might say "we are working on a billing and inventory management application that ties into our enterprise ERP system."

Wrap Up

So that's the majority of the content from my presentation on agile to my coworkers. The Slides have a bit more information if interested.

In my general experience agile is a concept worth learning about. The teams I have been on have had higher quality products, more tightly knit relationships within the teams and have ultimately delivered a result that the business was very happy with. Agile may not work in every environment but if you can make it work in your organization I would recommend giving it a try.

Intro to Agile

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

http://agilemanifesto.org

Last Thursday I had the opportunity to present agile development to most of my team mates at Paragon. For some of them it was the first time being formally introduced to the concepts  and practices and I'm happy to say it seemed to go really well. I promised a followup post on my site summarizing what was covered and providing a copy of my slides. This is part one of that post. I will provide a copy of the slides on both posts but for this first post I'm only covering the first half of the presentation.

Agile is a state of mind

There's a misconception among a lot of developers that agile is the sum of the practices, tools and processes that make it up. In other words to many people agile is doing test drive development, pairing and working in sprints. The problem with this view is that it really misses the fundamental reason why agile came to be. In fact when the agile manifesto was written it explicitly included the phrase "Individuals and interactions over processes and tools."

Agile is about how we interact with our team and our customers. Agile is about how we embrace change. The tools we use to accomplish those goals are simply a means to an end.

There are four points that, to me, making up a successful agile team.

Delivering the right product at the right time

The right product is composed of the product that delivers the most business value to the customer. Agile teams do this by allowing the business to prioritize all the work and performing that work in priority order.

The right time is when the remaining tasks no longer convey a competitive advantage to the business. We identify when this is by showing the product to real people (users, business, stakeholders) as early as possible and acting on the feedback we receive from those people.

Focusing on Communication and Collaboration

Agile teams work in close quarters. All the members of the team (project management, development, testing and the business) should all be in close physical proximity if at all possible. This allows the free flow type communication that teams thrive on. It is true that remote teams can accomplish great things in an agile manner but collocated teams are preferred.

Agile teams work with an embedded business representative. Ideally this individual is an actual employee or stakeholder of the company the product is being built for. Unfortunately that isn't always possible. In their absence any individual who can make an authoritative decision on the functionality of the application and who can set task priority can represent the business. Regardless the customer needs to be involved in key planning meetings and demos.

We strive to create short feedback cycles. This means developers working closely with testers to test the code they write as it is completed. This also means working with our business customers to show them what we build as we build it and getting feedback on it. Short feedback cycles allow us to act on problems or inconsistencies and respond to shifting priorities quickly. "Throwing the code over the wall" to QA is no longer acceptable.

Finally we learn from the past. We constantly evaluate how we are doing and adapt quickly to make sure the project is successful.

Embracing Change (even late in the process)

Agile teams deliver what the business needs and not what they think they want.

Traditionally changing the requirements of a project in midstream is frowned upon. Its scope creep or against the design and makes our jobs as developers more difficult. We introduce elaborate change management processes to punish any poor business person who should dare change our requirements.

Agile teams move past that mentality. We recognize that as we build our customers product their priorities will change. As they see the progress we are making some items will become higher priority than other and sometimes entirley new areas of functionality will become apparent.

The result of embracing change is a happier customer and higher acceptance of the product. The real goal though is delivering the right product. We know its the right product because the customer has been involved the entire time, has shifted priorities as necessary and has told us that there is no more work to be done to deliver the benefit they need.

Deferring decisions till the last possible moment

We can't predict the future but we love to try. Agile teams defer decisions until they have as much information as possible to make the right one. Very rarely is this the beginning of the project. This means we generally don't write comprehensive design documents or requirements documents well in advance of the development process because we don't know enough to accurately write those documents.

As developers this means only introducing software design elements we need. Agile has a phrase for this - YAGNI or "You ain't gonna need it." Generally if we need something once we build it in the simplest way that could possibly work, if we need it again we refactor our solution to be more flexible or anticipate future changes.

To be continued

So the message of the first part of this presentation is that agile is inherently philosophical. It's how we look at a project and how we interact with our team.

In the next post I'll cover the second half of the presentation, which goes over some of the practices that are used to help run an agile project. If you want a sneak peek here's a copy of the slides.

Into to Agile (PDF)