Intro to Agile

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

http://agilemanifesto.org

Last Thursday I had the opportunity to present agile development to most of my team mates at Paragon. For some of them it was the first time being formally introduced to the concepts  and practices and I'm happy to say it seemed to go really well. I promised a followup post on my site summarizing what was covered and providing a copy of my slides. This is part one of that post. I will provide a copy of the slides on both posts but for this first post I'm only covering the first half of the presentation.

Agile is a state of mind

There's a misconception among a lot of developers that agile is the sum of the practices, tools and processes that make it up. In other words to many people agile is doing test drive development, pairing and working in sprints. The problem with this view is that it really misses the fundamental reason why agile came to be. In fact when the agile manifesto was written it explicitly included the phrase "Individuals and interactions over processes and tools."

Agile is about how we interact with our team and our customers. Agile is about how we embrace change. The tools we use to accomplish those goals are simply a means to an end.

There are four points that, to me, making up a successful agile team.

Delivering the right product at the right time

The right product is composed of the product that delivers the most business value to the customer. Agile teams do this by allowing the business to prioritize all the work and performing that work in priority order.

The right time is when the remaining tasks no longer convey a competitive advantage to the business. We identify when this is by showing the product to real people (users, business, stakeholders) as early as possible and acting on the feedback we receive from those people.

Focusing on Communication and Collaboration

Agile teams work in close quarters. All the members of the team (project management, development, testing and the business) should all be in close physical proximity if at all possible. This allows the free flow type communication that teams thrive on. It is true that remote teams can accomplish great things in an agile manner but collocated teams are preferred.

Agile teams work with an embedded business representative. Ideally this individual is an actual employee or stakeholder of the company the product is being built for. Unfortunately that isn't always possible. In their absence any individual who can make an authoritative decision on the functionality of the application and who can set task priority can represent the business. Regardless the customer needs to be involved in key planning meetings and demos.

We strive to create short feedback cycles. This means developers working closely with testers to test the code they write as it is completed. This also means working with our business customers to show them what we build as we build it and getting feedback on it. Short feedback cycles allow us to act on problems or inconsistencies and respond to shifting priorities quickly. "Throwing the code over the wall" to QA is no longer acceptable.

Finally we learn from the past. We constantly evaluate how we are doing and adapt quickly to make sure the project is successful.

Embracing Change (even late in the process)

Agile teams deliver what the business needs and not what they think they want.

Traditionally changing the requirements of a project in midstream is frowned upon. Its scope creep or against the design and makes our jobs as developers more difficult. We introduce elaborate change management processes to punish any poor business person who should dare change our requirements.

Agile teams move past that mentality. We recognize that as we build our customers product their priorities will change. As they see the progress we are making some items will become higher priority than other and sometimes entirley new areas of functionality will become apparent.

The result of embracing change is a happier customer and higher acceptance of the product. The real goal though is delivering the right product. We know its the right product because the customer has been involved the entire time, has shifted priorities as necessary and has told us that there is no more work to be done to deliver the benefit they need.

Deferring decisions till the last possible moment

We can't predict the future but we love to try. Agile teams defer decisions until they have as much information as possible to make the right one. Very rarely is this the beginning of the project. This means we generally don't write comprehensive design documents or requirements documents well in advance of the development process because we don't know enough to accurately write those documents.

As developers this means only introducing software design elements we need. Agile has a phrase for this - YAGNI or "You ain't gonna need it." Generally if we need something once we build it in the simplest way that could possibly work, if we need it again we refactor our solution to be more flexible or anticipate future changes.

To be continued

So the message of the first part of this presentation is that agile is inherently philosophical. It's how we look at a project and how we interact with our team.

In the next post I'll cover the second half of the presentation, which goes over some of the practices that are used to help run an agile project. If you want a sneak peek here's a copy of the slides.

Into to Agile (PDF)

Tools and Techniques I Recommend to Clients

I've been doing full time consulting work for about a year and a half now and for the most part I love it. I get to go into new organization with different needs and concerns, meet new people and try and improve their project, process or just in general help contribute to their business needs. Every client is different but there are some universal tools, practices or concepts that most anyone can benefit from. When given the opportunity to express these areas of improvement here are the things I generally talk about. It should be noted that the tools on this list are mostly geared toward .NET as that's what most of my clients use but some of the concepts are universal.

Tools

 

1. JetBrains Resharper - This tool is first on my list because in terms of productivity gains from tooling Resharper may be the single best thing your money can buy. Microsoft has made some serious strides in bring refactoring and TDD support into visual studio but in my opinion they remain second class citizens especially when you compare the out of the box support with what Resharper gives you. To be honest Resharper could sell JUST search by class name (Ctrl+N) or Search by File Name (Ctrl+Shift+N) and i would buy it. The upcoming version is said to provide better support for ASP markup, HTML markup and Javascript and I'll be buying it on release day.


2. Balsmiq Mockups - Among the private or small-cap development community the concept of using pen and paper or intentionally "messy" mockups has become a bit more prevalent. Unfortunately in "corporate america" my experiences have been that the business still expects a mockup to look like the final product. This results in somewhat rushed layouts becoming law and UX suffers. Balsamiq is a great mockup tool for communicating the general flow of an application without setting expectations to the point where they are no longer malleable.


3. JetBrains TeamCity - I'm going to talk about Continuous Integration and having a build environment in a moment but in terms of tooling for the concept TeamCity is excellent. It's a fairly straight forward UI and eliminates a lot of the configuration "older" tools like ThoughtWorks CruiseControl .NET required.


4. ThoughtWorks CruiseControl .NET - Ok so i just took a shot at CC.NET by saying it required a lot of configuration, and that's true, but it's also the most mature build system out there for .NET. In my opinion (and limited experience) Microsoft's build offering can't hold a candle to the community support and stability of CC.NET. I've seen it used in single team shops and giant enterprises. Oh and did I mention it's free?


5. nAnt - This item may fall off the list soon in favor of Rake if Kevin's experiences stay consistent. nAnt is essentially the scripting language you will use if you use CruiseControl or TeamCity. You could use MSBuild and whatever platform MS set up for TFS but i find nAnt has better community support.


6. NUnit and RhinoMocks - I'll write more about these when I talk about TDD, but in my experiences they remain the best (and most stable) tools for testing on .NET. MSpec is an up and comer that could shift my direction.

 

Processes

This section is going to sound like an Agile To Do list but that's mainly because a lot of the practices that people think of as "agile" are actually just a good idea regardless of your actual development methodology. Some of these seem like no-brainers but it's amazing how many organizations still lack some of the most basic and most established concepts in software development.

1. Source Control - My first item is my most important. If you are doing file system source control, making copies of your application and praying to whoever you pray to that you don't lose something you are doing it wrong. There is no "but i don't like checking things out" or "but its so much work to maintain that repository" or even "but I'm the only programmer" your doing it wrong. You should not being writing a line of code without a repository set up and ready. You also have no excuse,there are cheap or free hosted options or you could always set up subversion, mercurial, git or even TFS and do it in house.
   
   
   
   • Distributed Source Control - I would recommend a system like Git or Mercurial for managing your source code over a system like TFS or Subversion (or god forbit SourceSafe.) All of those (except maybe SourceSafe) are great systems and offer a huge benifit especially if you aren't using source control yet, but today the latest and greatest (and for a reason) are the distributed source control systems. In short DVCS is better because it offers easier branching with less cost, less down time, your people can work away from the network and still have full history, commit and branching and if anyones repository gets messed up everyone on the team has a copy of it. There is a much smaller chance of failure.
   
   
   • Non Locking VCS - This is a HUGE pet peeve of mine and I still don't understand why some systems default to file locking (I'm looking at you SourceSafe and TFS.) When you lock files on checkout you ensure that no one else on the team can work on that file. That means if I need to make a change to fix a bug or add a feature I have to either email my change to a teammate and ask them to make it, mark the file as writable and change it anyway then hope i remember not to overwrite my changes when i eventually can check it in, or bug someone to check in their code in a potentially incomplete state so i can make my change. Its a huge pain in the ass and it slows me down. Also for every bug i need to fix i probably spend 80% of the time finding the bug and 20% fixing it. If i spend that 80% of the time, find out I cant change the code and move on to something else in the meantime guess what, I'm going to spend that 80% again finding it when the file is checked back in. HUGE productivity loss.
   
   


2. Some sort of project management approach - Preferably a flavor of agile, but in this category even waterfall is better than the ad-hoc "hope the project is done on time" approach. It's kinda remarkable but I still run into clients that don't really do anything with regards to project management. Essentially a business analyst or business person goes to the development team and says "i need this built by this date" and people start typing. Scope and priority changes but in a "your half way through this and now you need to stop and work on this" manner and not in any sort of controlled way. Code is lost, productivity falls and in general it costs the organization way more than it should.


3. Continuous Integration - CI, put simply, is the idea that you should have some automated way to compile your code, execute metrics and tests against it and provide feedback to the team if it is still in a good stable place. In some organizations all you have is the compile step, that's ok set up a CI server anyway. It's worth it. People have a tendency to check in bad code when no one is watching, if the build goes red they know someone is watching. On a related note my next CI server will be named B1g Br0th3r


4. Test Driven Development - Books have been written about this topic but basically TDD is the notion that you should write a test for your code before you write your code, then write the code to make that test pass. It ensures that every line of code you write is adequately tested on a logical level (it doesn't get you out of manual system testing but it should cut the time and resources needed for that if done properly.)


5. Automated UI Testing - This one is a bit controversial as of late but I'm still recommending automated UI testing if a client has a technical QA team member who can maintain the test. These type of tests essentially drive the UI of your application to attempt to determine if it is executing as expected. They ARE fragile (software changes will drive changes in these tests) but if done by your testing team they provide a good feedback loop between development and QA. They also hit a layer that TDD usually misses. I would NOT recommend paying for tools, if your doing web dev Selenium and Watin are great and free. Also for a group with no TDD experience sometimes this type of testing can be an easier addition to an existing codebase but please don't get comfortable, learn TDD.

 

The Rest

Some quick props and a disclaimer.

Redgate SQL Tools - Best SQL Server Toolkit for DB types.
Beyond Compare - A great file comparison tool for windows.
SnagIt - A great Screen Shot application for windows.
AgileZen - An excellent online Kanban board build by a NorthEast Ohio local. A lot of people love pen and paper for Kanban but if your working with a distributed team AgileZen is the best I have seen.
Basecamp - Another great project management tool by the inventors of Ruby on Rails.

Disclaimer: Redgate, Jetbrains,  Scooter Software, Techsmith and Balsamiq all provided swag for Technic picnic I held last summer. That said their products rock and that's why I contacted them asking for a handout in the first place.

Live to Code

For Christmas my wife bought me a copy of Michael Symon's new cookbook Live to Cook: Recipes and Techniques to Rock your Kitchen. For any non Clevelanders and Food Network aficionados Symon is the head chef and owner of several successful restaurants in Cleveland and Detroit and is also an Iron Chef on Iron Chef America.

Now, I don't consider myself a chef and I don't regularly read cookbooks. In fact the gift may have been a not to subtle hint that I need to bone up on my culinary arts. That said I have taken to Cleveland and enjoy reading a bit about local people doing what they love and the various landmarks the city offers so I opened Symon's book and found something slightly unique. Hidden between the various recipes and techniques are some short descriptions of his inspiration as a chef. What drives him to do what he does and how he just loves to cook. It got me thinking about programming and some conversations I've had with friends in our local community.

In the book Symon speaks of passion, and how in his career thus far the most successful people he has worked with haven't been the best trained or the people with the most experience on the line but rather the people who were most passionate at what they do. It's an observation that carries through to programming and I imagine any other career where people take pride in the craftsmanship of what they do. In my, admittedly short so far, career I've had the opportunity to work with some incredibly talented and bright people and they all had one characteristic in common, they were passionate. You could get them talking for hours ad nausium about software and what makes clean code. How to best structure an algorithm or what new and exciting technique they were learning now.

In the book Symon tells a story of his first restaurant after school. He recalls how it was a small  30-40 seat place and he worked under a head chef who was self-educated. Despite the individuals lack of "formal" training Symon speaks to how much he learned from the man and how he gravitated to him as a mentor because of the passion he showed. His examples are things like an insistence on using fresh garlic and not pre-peeled garlic despite the hours of prep time required and manually roasting peppers instead of purchasing them preroasted. In other words, the details are what mattered to this individual and Symon maintains that he adheres to those same concepts in his restaurants today. This to me is analogous to the Test Driven and Behavior Driven Development movements. In our profession we spend time identifying the details of an application in advance. Planning it out in essence by writing tests against how the code should function. By doing so we ensure a cleaner more quality driven end result.

From a business perspective the analogy continues. In software you have your McDonalds and your Lola's and there really is nothing wrong with being either one. The key is in knowing who you are. If you want to be the Lola of the software world you're going to be focused on quality, delivering the best product your customer asks for and your work will demand a premium because of it. If you're McDonalds then your going to deliver quickly and afford-ably but it may not be as good as the premium competition.

That said I'd much rather frequent Lola than McDonalds.

Sometimes ASP.NET Baffles Me

Fair warning, this post is going to be a bit of a rant about ASP.Net and how it does nothing like every other web language on the planet.

So far today i've spent about six hours trying to come up with a solution to one of those problems that seems like it should be oh so simple to solve. As of this point, I got nothin'.

So the client I'm at these past couple months uses a third party set of ASP.Net user controls to handle a lot of AJAX tricks and visual prettiness. The only problem is from what I can tell they don't work. They don't render correctly in Firefox, occasionally don't render correctly in IE and cause some serious issues when paired with jQuery. The problem i was tasked to solve was a scenario where when using this third party controlset's popup box (essentially a floating DIV) it could cause the rest of the content on the page to fall down and be essentially invisible. To make matters worse the popup itself showed up below the fold, the scroll bar was disabled and it was modal, so there's no way to get back to a usable state short of refreshing the browser. After a couple hours playing with the control I decided it would be easier to pull the control out completely and go to the quite excellent jQuery UI dialog plugin. Once I applied it to the mischievous popup the UI worked and all was well... or so i thought.

A day passed and our tech lead came over. He asked if I could convert the rest of the popups in the application to jQuery for consistency and so that the issue wouldn't reoccur in the future, so I started doing that. Things were going well until I ran into another of this toolkits controls, CallbackPanel. For all intents and purposes CallbackPanel is like an update panel except it is invoked directly in javascript. I started applying the jQuery work to a popup containing one of these panels and noticed an odd trend. It behaved correctly when executed the first time, but when run again without a page refresh the javascript would execute but the server side code was never called. As of yet I have been unable to resolve this and have a pending support ticket with the vendor. My theory is the event is somehow tied to where in the DOM the panel original resides. When i put that panel in a jQuery dialog it is updated in the DOM and that breaks the event.

Since i have little faith in the vendor's speed to fix this (or tell me its intended functionality) I opted to move ahead and try and migrate the whole section into a simple AJAX call back. I would make a call to a WebMethod and return some markup from the user control that would then be injected into the popup and voila.. ajaxy goodness. This is how i would approach this on just about any other platform. Alas, .NET is not any other platform.

The first issue i run into is that WebMethods have to be static. That means I cant access the usercontrol already on the page.

Ok no problem, I'll instantiate one and get the rendered HTML, pass that back to the client.

Try instantiating the class, no go. Turns out you need to use LoadControl (a non-static method of course) to create an instance of a user control otherwise all the child controls don't get instantiated.

Crap, ok Mr. John Miller find this article for me. I create a page object, load the control, use reflection to set various properties and even isolate all this into a class called UserControlRenderer for reuse. Things are looking good until I run it.

Error executing child request for handler ‘System.Web.UI.Page

Son of a bitch are you kidding me? Do a bit of digging and find out that you cant render a DataGrid (or GridView) unless it is contained in a form with runat=server set. Which means this approach of rendering just the user control to a string and returning it won't work for my implementation because I'm using one of the most common controls in .Net.

All told I have about 45 lines of code to do nothing but try and render the control (Set up, instantiation, the class that does the rendering, etc) and I have to tap into reflection to do it. Oh and it doesn't even work.

For those of you that haven't played with any languages outside of .Net here's how much code I would need in say Ruby to render a single "user control" (they are called partials in rails)

render :partial=>"mycontrol", :layout=>false

One line... one line of code. And it works.Of course I have to get the data and set it to a variable accessable to the control, then render out the html... but I have to do all that in .Net too and I didnt count those lines in my 45 above.

From day one the ASP.Net team assumed that their target audience was a group of people who didn't understand the web. Didn't get the general stateless nature, or how HTML works. That probably made a ton of sense in 2001 when .Net was young, people were used to writing VB windows apps and there was a push to migrate those application to the web. It 2010 it makes no sense. If you are web developer and don't understand how the web works I dare say you are a quite incompetent web developer. Thank god for the push to ASP.Net MVC, here's hoping it takes over in enterprise soon.

Ok rant done. Next post will be a bit more productive.

My 2010 Challenge List

Last year my friend John posted about challenging himself for the new year. Obviously not a new concept but there was something I enjoyed about his posting the ideas publicly and then following up on them, if for no other reason than I realized I too wanted to accomplish some of the same things. So in that same vein I figured what better way to dust off this blog than to commit to digital pen and paper my list of resolutions. In fact the dustiness of the blog will directly influence my first one:

Professional Goals:

1) Update this blog with a new topic at least twice a month. This seems an obtainable goal, I do have new ideas to talk about I just never follow up or write them down. Time to change that.

2) Speak at something, anything, and codemash doesn't count. I consider myself a terrible public speaker. Always have been. I get all nervous and shaky and well.. its awkward, but every time I step up and do it I get a little bit better. So I need to do it more often. This could be at something like Ignite Cleveland or a user group or conference. Whichever.

3) Continue developing a Kanban approach for consultancies and apply it to my current job. Towards the end of this past year I started getting my coworkers interested in improving how we do project work. We started using a Kanban-lite style approach for in house work and overall people seem to like it. I want to build on this and actually teach people how the system works. Fortunately I have some fellow advocates in this one.

4) Learn some of the new development techniques and technologies in the .NET world. My day job is 90% .NET work but in reality I find myself more passionate about open source technologies and little scrappy "upstart" languages like Ruby. That said there is some really cool stuff happening in .NET and in order to continue to make my career there I need to learn them. So a short list:

• NHibernate w/ Castle or ActiveRecord


• WPF


• WCF


• Microsoft MVC

5) Continue to get to know Ruby. I really like this platform and the communities focus on TDD (or BDD) and generally maintainable code is commendable. This year I mainly want to accomplish a couple things.

• Use cucumber on a project


• Get comfortable with Rails and RSpec where I don't need a book to look up simple things anymore.


• Compete and complete an application for Rails Rumble 2010


• Relaunch EventCasts (the new name for our previous Rails Rumble app Twoups.

6) Get things squared away with Greenfield Studios. Greenfield Studios is the company that some friends and I started last year to try and build a site that would help people affordable use and locate green technologies. The site never amounted to much due to a lack of real data on the topic so it has effectively been shuttered. Since then I've wanted to reuse the name and restructure it legally to serve as an identity to put on any independent contracting work or product work I do. This means finding a good lawyer who I can afford. Suggestions are welcome :-)

7) Finish EventCasts and one other side project. Lately I've been using google wave to track the various side project ideas I have. At present there are 4 in the hopper. I want to get EventCasts done and one of the other 4 by the end of 2010.

House Goals

1) Finish remodeling the basement (without hiring someone to do it). A year ago when we moved in I decided I wanted to build an entertainment / theatre room in my basement. Not sure I'll be able to afford the expensive projector for it this year but I can finish the remodeling part. Also i want to learn how to do this stuff, so hiring someone for anything but the electrical and maybe carpet is off limits.

2) Seal the floor in the garage - I like the look of a painted sealed floor.

Personal Goals

1) Take a vacation somewhere outside of Ohio or any state that borders Ohio. I haven't gone away on vacation (excepting our honeymoon) in a long time.  My wife and I really want to go to NYC or Vegas, which neither of us have ever been to.

2) Pay off my friends at Discover Card. I bought a lot of dumb crap in college and continue to do so.

3) Figure out a "diet plan" and stick to it, lose 10 lbs a month. This is quite possibly insane, and I probably wont pull off 120 lbs in a year but what the hell why not shoot for the moon. I hate the phrase diet because even I know there's no such thing as a fad diet that works. Its about discipline and consistency in eating, exercising and life style. However despite knowing this academically I have yet to actually accomplish it.

I suppose that's all for now. I liked John's reading list from his post but I'll save that for another update (2 per month now right?)

Do you have any plans for 2010 or do any of these "resolutions" ring true for you as well? That's what comments are for :-)

What is the web missing?

This week's announcement of an entirely web-based operating system in Google Chrome OS has sparked a lot of discussion. Google's strategy for now seems to be to invade the netbook space, an area where an offering like Chrome OS could do well. Netbooks, by their very nature (and name) are designed around internet use and there are very few day to day activities that you can't do in the "cloud." The unanswered question is where google will go with this product after netbooks. They have alluded to branching into the desktop market but there are still a number of areas where a purely internet based OS won't work, for now.

Since reading about C.O.S. I have been wondering what market opportunities are out there if suddenly the web becomes the operating system. What areas of computing still demand a rich desktop environment over a web application and what could be done to move past that. A few things came to mind.

First one that is near and dear to my heart - software development. Right now us code monkeys do most of our work in an editor/IDE running code locally on our machines before promoting it to some sort of shared environment, be it a testing environment or for the more radical straight to production. The fanciest of us use tools and practices like XUnit, BDD and various code metrics applications. Right now there is no purely cloud based implementation of this. In order to move software development onto the web you could need some combination of a hosted virtual development machine with a web interface and the appropriate tooling to make that work. We would need a web-based programmers text editor (yes i know people have ported VI to javascript.) Also I'd like to wish visual studio .NET and eclipse the best of luck in converting their UI to a website.... hah.

The next area of computing that came to mind is a bit more consumer oriented - music. Yes Pandora rocks. So does Last.FM, but their service isn't really the same as say iTunes. In order for the web to become THE computing platform we would need a hosted service where you could listen to your music library, make playlists and browse new music. Also... and here's the fun part... we have to figure out how to sync this to an iPhone / iPod / Zune / whatever.

The last, and possibly most obvious one, is gaming. A service like OnLive would help but who knows when that will be out.

So what other areas of computing currently need a traditional OS?

Book Review: The Passionate Programmer

Chad Fowler's The Passionate Programmer is a book about happiness. It is about how to lead a successful happy career doing something that you really enjoy. Naturally it is geared toward software programmers but in reality many of the tips Fowler hands out could be applied to almost any career.

Fowler takes his experiences as a programmer first and a musician second and uses them to identify a set of 53 "tips" broken up into 5 main sections. These tips run the gamut from identifying a core technology or specialty to going out on your own as an independent.

At first glance much of Fowler's content could be pushed off as common sense, however in many cases the most obvious of concepts isnt readily apparent until it is brought to one's attention. The notion of treating one's career as a product is a popular notion that Fowler subscribes to and bases his advice on.

Fowler begins with "Choosing Your Market", a segment focused on identifying a technology or specialty that fits your career goals. He identifies criteria to consider when evaluating the investment required to become versed in a particular technology.

Next he lays out "Investing in Your Product." This section identifies how one can develop expertise in both the technologies they have chosen to specialize in as well as how to become a generally excellent programmer.

"Executing" is the next section of the book, which identifies how you can work to the best of your ability in your day to day tasks. A good percentage of this section is motivational in nature. Chapters like "How Good a Job Can I Do Today?" really seek to overcome the desire to stay in bed in the morning.

Fowler then covers "Marketing..." with tips about how to get your name out both within your current place of employment as well as the greater community.

Finally the book finishes with "Maintaining Your Edge" a call to action to not give up on the prior tips and to continue to treat your career as your product.

Overall the book is a fascinating read and well worth the investment for anyone who enjoys the "softer" side of software development. It serves as an excellent companion to The Pragmatic Programmer, a book with whom it shares some lineage by virtue of the publishing house.

On a school scale I would give The Passionate Programmer a solid B+. It was a good read, with some great information, but it isn't a book I would be compelled to buy as a gift or to put on a "required reading" list.