April – 4th Agile Principle

Business people and developers must work together daily throughout the project.

We are now getting to one of the most widely ignored principles of agile development. It is clear, succinct and quite definite. It says that business people and developers must work together daily. Note that it doesn’t say business people and developers should work together daily. The keywords are must and daily. Why is this principle often ignored? Because business people are often too busy to talk to developers. There are several reasons for that. First of all, they usually have other responsibilities than managing software development. Second, they may simply not be available every day. Third, business people and developers often don’t understand each other. It’s almost as if they speak a different language. Developers speak about technology and business people speak about goals, opportunities, and strategies.

Before we go into these issues, let’s briefly elaborate why it is important that developers and business people cooperate on an ongoing basis. Unlike the waterfall method or any other method that is based on a grand plan, agile development relies on a progressive formulation of requirements. This is the same feature that makes agile development adaptive and open to change. The drafting of requirements happens incrementally and thereby mirrors the development process. Specifications evolve throughout the project. There is no single prescribed method for how business requirements are being translated into detailed technical specifications. It can be done in a number of ways. However, it is understood that business people and developers must be in an ongoing dialog to make this happen.

Frequently, requirements begin with a high-level road map or whitepaper. This document contains a description of the overall goals and high-level functionality of the project. It is generally too abstract to be used for technical implementation. However, it may be followed a description of the envisioned architecture. The next step consists of defining different large-scale features (epics) that touch upon distinct aspects of functionality. This step again requires a dialogue of business people and developers. Epics are still too abstract and too large to be worked on directly. The entire process of requirements gathering can be seen as a successive refinement of specifications to the point where specifications can be encoded in computer language. Epics are broken down further into units of work that agile teams consider appropriate for concrete implementation, so-called user stories.

A user story is a conceptual tool of agile development. It describes a software feature from the perspective of a user. For example: “As an administrator, I want to be able to approve new comments before they are posted.” This is a proper user story containing a user role and a feature description. During the course of technical planning of a user story, further details are specified, such as the user interface particulars for that specific story. Diagrams, tables and UI mockups may be created during this phase. This implies again that there is dialogue between those who provide specifications (business) and those who implement them (developers). Without continuous cooperation, requirements end up being too vague and software artifacts end up being incomplete or flawed. The requirements refinement process runs parallel to implementation and testing. It requires management from the business side. Together, they make up a sort of pipeline that runs from high-level specifications to usable software.

In a typical Scrum team, there is an appointed manager (called “product owner”) who leads the requirements refinement process. The product owner acts as a sort of intermediary between business and development. On the other hand, a typical Kanban team is cross-functional and has no appointed roles. Members from business provide guidance directly without a project manager. Project management is often handled on demand by individual team members, depending on the nature of the task. In a Crystal team, project management depends on team size. There may or may not be an appointed project manager, often an executive sponsor. Business and development come together for Blitz planning sessions to define tasks and schedules. The refinement process is informal and relies on other methods such as co-location, daily meetings, or brief review-and-feedback loops with expert users. The problem of translating business requirements into software features and codeable functions can be solved in many ways. The one thing that all the different agile methods have in common is that business people and developers work together on an ongoing basis.

What can be done to improve communication between business people and developers? The problem of allocating sufficient time for exchanges can be solved in different ways. Appointing a product manager who works face-to-face with the team works well. Setting up regular meetings, such as feature planning, iteration planning, reviews, and backlog refinement meetings also works well. Having an executive sponsor working with the team is likewise a good solution if the sponsor is approachable and dedicates enough time to the task. Even informal communication can succeed if business people and developers are co-located at the same work space. The language problem can at least partly be tackled by developing a domain-specific vocabulary. Organisations usually have quite specific structures, roles, workflows, service offerings, etc. It is important to agree on a fixed terminology for these when phrasing requirements. This helps eliminating ambiguity, because ambiguity and vagueness are the principal obstructions to a successful implementation.

In summary, the 4th agile principle describes one of the most important factors to determine success or failure of software development projects.

January – 1st Agile Principle

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

The year has 12 months and agile development is based on 12 principles. Today, let’s start with the first agile principle. It is about customer orientation, perhaps the main motivation for agile development. Some of you might know, that it is now 18 years ago since a group of software engineers came together in Snowbird, Utah for a brief winter conference on project management. The result of that meeting was the well-known Agile Manifesto. It had a big and lasting impact on the software industry. Somewhat later the Agile movement started out and today, agile methods can be considered mainstream. However, back then things were different.

At university and during the early years of my career I learned that software development is divided into six stages. It starts with requirements analysis followed by three production phases, followed by deployment and maintenance. It was understood that the software development lifecycle (SDLC) phases are strictly linear and sequential. One is completed after the other. The customer is only involved in the first and the two last of these phases. Software companies spent enormous efforts on analysis, planning and documentation in order to get cost and time estimates right. Customers had to be enormously patient to wait for functioning software and they had no say in its production.

This process carried significant risks and turned out to be frustrating for both parties. Although customers initially received detailed contract specifications and a fixed price offer, the product was regularly late and below expectations. Even worse, some requirements changed during implementation and thus the software was already outdated at the time of delivery. Another costly change order was then inevitable. Things weren’t much better for developers. Unforeseen events and technical difficulties led to exceedance of cost and time estimates. In the worst case scenario, these circumstances left the developer unable to fulfill the contract and pushing towards bankruptcy. The client received either no product, or an unusable product for their investment.

In 2001, I was managing my second startup company and had no intention of repeating past mistakes in project management. Like many others at the time, I used a homegrown set of lightweight methods for SDLC management. I had acquired these during the late nineties in my practical work with different clients. I knew nothing about the Agile Manifesto at the time, but I found that the traditional waterfall approach didn’t work for us. Instead, I visited my clients every two weeks. During a long meeting which often took the larger part of a day, I would first present the new features that my team and I had developed. I would then install the update on my clients’ servers (this was well before the time of Internet-based deployments). Finally, I would discuss the most-wanted features and changes for the next cycle. There was an understanding that if the client found any bugs, they would be fixed immediately within a single release cycle.

Why did this work so well? Because the customer was always kept in the loop. They received usable software every two weeks that provided actual business value. They could control and even participate in the design process. They could correct mistakes early. Last but not least, their financial commitment wasn’t burdened by huge risk, because they paid monthly installments. This way of working puts the client in control and keeps them happy. And this is exactly what the first agile principle is about. I learned about agile project management only some years later and I found it to be an excellent fit for what we were doing. It confirmed our practice as a development company and it also broadened and enhanced our methods.

The reason why the this principle is the first, is given in the beginning clause: our highest priority is to satisfy the customer. Fundamentally, the agile methodology is about customer satisfaction. Normally, the customer is thought of as a third party outside the software company, but it could very well be the same company doing in-house development. It doesn’t matter. In the latter case, the customer role is filled by another department within the organisation. It is important to note that neither technical goals, nor abstract principles of management provide the foundation for agile development, but business goals. The middle part of the sentence mentions early and continuous delivery. This makes perfect sense from the perspective of developers, because software artefacts evolve and grow in small incremental steps. It also makes sense from the perspective of the customer, because they can begin using individual features before the entire package is completed.

Even more benefits are reaped by early and continuous delivery. The customer can control and prioritise the development of individual components. Mistakes can be corrected cost-efficiently at an early stage. The customer can verify or invalidate concepts in practical application. Developers are able to create proof-of-concepts without committing to a specific design or technology too early. Finally, the first agile principle speaks of the delivery of valuable software. The value of software is what clients and developers ultimately aim to realise together. This always translates into some sort of business value. For example, the software might enable the client to respond faster to their own clients. Or it may automate some task which previously had to be done in a tedious manual way. Or it may enable the client to gain deeper insights into markets, opportunities, and risks. The possibilities are endless. The sooner the software enters production stage, the earlier the customer can reap its benefits.

In summary, the first principle encapsulates the motivation and mission for agile development. It’s all about customer satisfaction and providing business value through software.