May – 5th agile principle

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

All agile principles are about the “how” as in “how to get things done”. If there is any principle that touches upon organisational culture, it is this one. We can ask: How do people in an agile team interact and contribute to an organization’s goal? What are the underlying beliefs, values and assumptions of an agile team? So, the fifth agile principle is about the “how” of project management and organisational culture.

There are three propositions in this statement. The first one is to build projects around motivated individuals. Notice that there is no mention of the words team, management, or leadership. The emphasis is instead on individuals and motivation. Individuals carry out tasks and take responsibility. Motivation is the force that propels them. Successful completion of a project is not the outcome of good leadership, good management, and not even a good team. It is achieved because motivated people who work together.

The topic of motivation is very important. At the same time, it is left open where motivation comes from and how individuals are motivated. They may be motivated because they share the same ideas ideas and goals. They may be motivated by the challenge that the projects presents. They may be motivated, because their work allows them grow their understanding and abilities. Or they may simply love the work they do. Whatever the source of motivation may be, it is key to the success of the project. Motivation is also contagious. If team members are highly motivated, they create an environment in which it is hard to stay unmotivated.

The focus on individuals is directly tied to the first of the four agile values: “Individuals and interactions over processes and tools.” Individuals bring with them not just motivation, but specific skills and experience that is valuable to others as well as to the overall project success. Individuals are integral parts of a team and are not easily replaced. They inevitably affect group dynamics and leave a personal mark on the work project. Which is why agile teams are often composed of multiple senior developers.

The second proposition is: “give them the environment and support they need.” To begin with, this includes the physical environment. A team obviously needs some sort of space where collaboration takes place. Traditionally, this is a shared office where people come together and communicate face-to-face. Increasingly, this is also virtual space that teams configure to their needs for online collaboration. There are a myriad ways of doing this. Suffice to say that physical shared space is preferred in agile project management. Physical space is conducive to informal and efficient communication.

Besides space, people need workstations, laptops, servers, equipment, software, tools, online resources and other “physical stuff” required to accomplish the given task. The term “environment” extends into the non-physical realm, however. A good work environment entails non-physical attributes, such as being safe, pleasant, respectful, rewarding, encouraging and stimulating. If you find that your work environment has all of these attributes, consider yourself lucky.

The support part of this statement plays in tandem with the environment part. A good environment is always supportive. Apart from financial support, this primarily means providing a platform and structure for cooperation. This could be the way a team organises its workflow, for example. But it is actually much more. Support can again mean physical and non-physical things. A good coffee machine and pleasant offices may be considered just as important as a well-maintained backlog and intellectually inspiring tech talks. At the top of the list, you are likely to find non-physical qualities such as transparency, recognition, and good work-life balance.

The third and final proposition of the 5th agile principle is: “Trust them to get the job done.” This piece of advice is most telling about the organisational culture that goes along with agile development. It is a radical departure from traditional top-down management. In a typical waterfall project, goals and milestones are prescribed by project leads and managers. The fulfilment of these goals is then ensured by implementing measures to gauge and control project progress. Or at least that is the idea.

In reality, both planning and measuring progress are often flawed. For example, the plan may contain a high degree of ambiguity, or the measurement does not regard code quality. By contrast, agile development operates on the assumption that the desired outcome (project success) follows naturally from providing the right conditions. This means, if there are motivated people working in an environment that is conducive to cooperation and if they receive adequate and sufficient support from management, then they will automatically get the job done. In fact, they will even get the job done in suboptimal conditions as long as they remain motivated.

The advise to trust people to get the job done comes from a place that has embraced a non-autocratic style of management. Responsibility and decision making is delegated to the individuals who work on a project. Managers assume the role of mentors and coaches rather than that of a dictatorial leader. Much emphasis is put on intrinsic motivation instead of extrinsic rewards and control mechanisms.

April – 4th Agile Principle

Business people and developers must work together daily throughout the project.

We are now getting to one of the most widely ignored principles of agile development. It is clear, succinct and quite definite. It says that business people and developers must work together daily. Note that it doesn’t say business people and developers should work together daily. The keywords are must and daily. Why is this principle often ignored? Because business people are often too busy to talk to developers. There are several reasons for that. First of all, they usually have other responsibilities than managing software development. Second, they may simply not be available every day. Third, business people and developers often don’t understand each other. It’s almost as if they speak a different language. Developers speak about technology and business people speak about goals, opportunities, and strategies.

Before we go into these issues, let’s briefly elaborate why it is important that developers and business people cooperate on an ongoing basis. Unlike the waterfall method or any other method that is based on a grand plan, agile development relies on a progressive formulation of requirements. This is the same feature that makes agile development adaptive and open to change. The drafting of requirements happens incrementally and thereby mirrors the development process. Specifications evolve throughout the project. There is no single prescribed method for how business requirements are being translated into detailed technical specifications. It can be done in a number of ways. However, it is understood that business people and developers must be in an ongoing dialog to make this happen.

Frequently, requirements begin with a high-level road map or whitepaper. This document contains a description of the overall goals and high-level functionality of the project. It is generally too abstract to be used for technical implementation. However, it may be followed a description of the envisioned architecture. The next step consists of defining different large-scale features (epics) that touch upon distinct aspects of functionality. This step again requires a dialogue of business people and developers. Epics are still too abstract and too large to be worked on directly. The entire process of requirements gathering can be seen as a successive refinement of specifications to the point where specifications can be encoded in computer language. Epics are broken down further into units of work that agile teams consider appropriate for concrete implementation, so-called user stories.

A user story is a conceptual tool of agile development. It describes a software feature from the perspective of a user. For example: “As an administrator, I want to be able to approve new comments before they are posted.” This is a proper user story containing a user role and a feature description. During the course of technical planning of a user story, further details are specified, such as the user interface particulars for that specific story. Diagrams, tables and UI mockups may be created during this phase. This implies again that there is dialogue between those who provide specifications (business) and those who implement them (developers). Without continuous cooperation, requirements end up being too vague and software artifacts end up being incomplete or flawed. The requirements refinement process runs parallel to implementation and testing. It requires management from the business side. Together, they make up a sort of pipeline that runs from high-level specifications to usable software.

In a typical Scrum team, there is an appointed manager (called “product owner”) who leads the requirements refinement process. The product owner acts as a sort of intermediary between business and development. On the other hand, a typical Kanban team is cross-functional and has no appointed roles. Members from business provide guidance directly without a project manager. Project management is often handled on demand by individual team members, depending on the nature of the task. In a Crystal team, project management depends on team size. There may or may not be an appointed project manager, often an executive sponsor. Business and development come together for Blitz planning sessions to define tasks and schedules. The refinement process is informal and relies on other methods such as co-location, daily meetings, or brief review-and-feedback loops with expert users. The problem of translating business requirements into software features and codeable functions can be solved in many ways. The one thing that all the different agile methods have in common is that business people and developers work together on an ongoing basis.

What can be done to improve communication between business people and developers? The problem of allocating sufficient time for exchanges can be solved in different ways. Appointing a product manager who works face-to-face with the team works well. Setting up regular meetings, such as feature planning, iteration planning, reviews, and backlog refinement meetings also works well. Having an executive sponsor working with the team is likewise a good solution if the sponsor is approachable and dedicates enough time to the task. Even informal communication can succeed if business people and developers are co-located at the same work space. The language problem can at least partly be tackled by developing a domain-specific vocabulary. Organisations usually have quite specific structures, roles, workflows, service offerings, etc. It is important to agree on a fixed terminology for these when phrasing requirements. This helps eliminating ambiguity, because ambiguity and vagueness are the principal obstructions to a successful implementation.

In summary, the 4th agile principle describes one of the most important factors to determine success or failure of software development projects.

March – 3rd Agile Principle

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

You might wonder how the third agile principle differs from the first. The first principle talks about “early and continuous delivery” of software. This sounds a bit similar to “delivering software frequently”. However, there is a significant difference. The first principle is all about customer satisfaction. It is realised by continuously delivering value to customers. The third principle is all about method. The principal method used to achieve the goals of the first principle is development in iterations. An iteration is a defined time span during which the entire software development life cycle (SDLC) is traversed. An iteration starts with requirements analysis and ends with software delivery. It can be modeled as a closed and repeating cycle.

If we recall the waterfall model, there are some major differences. Mot importantly, the waterfall model traverses the software development cycle only once. This usually takes a much longer time. It is common that the duration of this long-term traversal cannot be predicted exactly. For that reason, deadlines are often postponed. Another major difference is that roll out/deployment and maintenance phases are periods at the end of the waterfall SDLC, whereas agile methods do not stipulate predetermined phases for these activities at all. Agile teams carry out maintenance “on the go”; it is part of the normal development work.

The time frame of an agile iteration is more rigid, by contrast. It is commonly fixed and takes anywhere between a few weeks and a few months. For example, teams that use Scrum often work in cycles of 2-4 weeks. There is no delay or postponement. Instead, if a feature is not ready, it is moved to the next iteration. The quantity of work completed within this relatively short amount of time is of course much smaller. So, at the beginning of each cycle, the team continues to build upon the foundation laid in the previous cycles. The process is therefore also incremental. This has certain advantages for both developers and clients.

Let’s state again that the agile process is both iterative and incremental. This is important. At the end of each iteration, new features and fixes are demonstrated to the stakeholders. In many cases, the iteration also results in a release, which means that software is deployed to production at the end of a single iteration. However, this is not written in stone. For example, a team may decide to work in 2 weeks iterations while adopting a release cycle of 4 weeks to match maintenance windows. Another team may prefer a continuous delivery model where features are released to production as soon as they become available.

It does not matter, as the iteration is not strictly coupled to the release plan. What matters is that software is delivered frequently and that the time span of an iteration is kept as short as practicably possible. This makes for a short feedback loop which allows stakeholders to intervene quickly. It increases the control of business stakeholders and allows for an amount of flexibility that is impossible to attain with the traditional waterfall process. For example, change requests and bug fixes can often be implemented and released within a single iteration making for a time to market that can simply not be achieved with any other method.

From what has been said, it follows that the agile process is also evolutionary and adaptive. Because large scale features are built incrementally over the span of multiple iterations, developers build on the foundation of existing architecture and functions. New blocks of software are added to existing blocks. Existing blocks are sometimes refactored or changed. It is therefore possible to test concepts in practice before a feature is completed. It is also possible to improve concepts and functions over the span of multiple iterations. The end result is always working software.

This means that software features, especially large scale features (called “epic” in Scrum methodology), are not typically conceived at the drawing board. Instead, prototypes and proof-of-concepts make their way into working software long before the planning and implementation of a large scale feature is completed. This is another major difference to the waterfall model. The key advantage provided by a short feedback loop is that bugs, logical errors and flawed concepts can be spotted and corrected early. This keeps the cost of such improvements to a minimum.

Because the evolutionary and adaptive nature of the agile software development process, planning and design is often restricted to a relatively small time frame and is open to change. For example, the requirements for implementing an epic could change midway during one of the many iterations it takes to complete the epic. In some cases, these requirement changes necessitate rethinking and altering existing features or even rethinking architecture. Code refactoring, the restructuring and improvement of existing code without changing its functionality, is therefore very important in agile development. Without continuous refactoring, agile projects are in danger of accumulating technical debt.

Continuous refactoring is perhaps the price that has to be paid for the extreme flexibility and quick time to market that agile development affords. A team that follows an adaptive approach must guard against code smells and software entropy. Only with sustained attention to these problems, software will stay maintainable and extensible in the long run.

The Open Closed Principle Revisited

An interesting discussion came up in a recent code review. Looking over the OOP implementation of a menu class, a colleague remarked that classes should always be designed open for extension and closed for modification. Sounds right, as it reflects generally accepted OOP practices. The open-closed principle, the “O” in SOLID, is an essential principle of sound object-oriented design. You probably already knew that. Here is the initial design of that class.

class Menu {
private constructor()

static createForUser(user: User) : Menu
static createForLegacyUser(user: LegacyUser): Menu

public getThisMenu(): MenuData
public getThatMenu(): MenuData
public getAnotherMenu(): MenuData

The menu class can be instantiated in two different ways, either with a regular user or with a legacy user. Both user objects provide authorization via ACL. The menu class has several getters that return different menus. The dynamically created MenuData contains only the menu options the user has access to. The private implementation methods are omitted. These perform authorization using the user objects and also translate the menu items into the user’s UI language. The menus themselves are configured in external YAML files and their structure can be changed as long the expected YAML format is maintained.

As you can gather from the design, this class is not designed with extension in mind. It is possible to add other menus, but only by extending it and adding a getYetAnotherMenu() method. Unfortunately, inheritance is not very practicable here. Extending for other types of users isn’t possible at all, because the respective authorisation code would have to be added to the implementation. Let’s summarise:

  1. Very easy: adding or removing menu options and sub menus to the existing menus
  2. Possible, but not that easy: adding or removing menus
  3. Not possible: adding other types of users

An obvious weakness of the above design is the redundancy in the getters and the fact that the menus themselves are fixed. This can be improved quite easily:

class Menu {
private constructor()
static createForUser(user: User) : Menu
static createForLegacyUser(user: LegacyUser): Menu

public getMenu(menuIdentifier: string): MenuData

Instead of invoking it with menu = Menu.createForUser(user).getThisMenu() to obtain menu data, this class is now invoked with menu = Menu.createForUser(user).getMenu(‘thisMenu’). The identifier is associated with the respective YAML file from which the menu data structure is created. Let’s summarise again:

  1. Very easy: adding or removing menu options and sub menus to the existing menus
  2. Very easy: adding or removing menus
  3. Not possible: adding other types of users

At this point, I was quite happy with the simplicity and flexibility of the design. We’ve made it both easy to extend menu options as well as menus themselves. My colleague, however, wasn’t happy. He insisted that the open-closed principle must also be applied to user types. And this is where it gets philosophical. Of course, it is possible to do that. But it requires significant changes and a further abstraction. We need to remove the authorisation implementation from the menu class, create an Authoriser interface and provide Authoriser classes for regular users and legacy users that implement Authoriser.

class Menu {
static create(authoriser: Authoriser) : Menu
public getMenu(menuIdentifier: string): MenuData

interface Authoriser {
boolean authorise(role: AclRole)

class RegularUserAuthoriser implements Authoriser {
constructor(user: User)
authorise(role: AclRole)

class LegacyUserAuthoriser implements Authoriser {
constructor(user: LegacyUser)
authorise(role: AclRole)

Why philosophical? Because, while this is a textbook implementation of the open-closed principle, we have never added any other user types in years. The system has only two types of users: regular users and legacy users. The likelihood of having other types of users in future is almost nil. This raises two questions: First, is the cost of extensibility and extra abstraction justified? This change obviously makes the code more complex. Second, how do you know what kind of extensibility will be required in future?

I would answer the first question with “no”. It is not worthwhile complicating the design to achieve extensibility if this extensibility is not needed. Doing so would constitute a YAGNI antipattern. YAGNI stands for “you aren’t gonna need it” and originates from the core principles of extreme programming (XP). According to XP, features should be implemented as simple as possible and without making unwarranted assumptions about future use cases. This is very much related to the KISS principle, which in turn has its foundation in philosophy: Occam’s razor and the law of parsimony. In practical terms: the simplest implementation for a given requirement is the best implementation.

If you paid attention, you might already see the dilemma. There seems to be a contradiction between YAGNI and the open-closed principle. If we design a class that is open for extension and closed for modification, this usually means we have to define additional interfaces that formalise the extensible behaviour. This is to say, the open-closed principle results in increased complexity, contradicting KISS, YAGNI and Occam’s razor. So, what should we do?

Consider the second question we have asked before: how do you know what kind of extensibility will be required in future? Simply speaking, if you don’t know, then don’t build it. Classes usually contain more than a sinlgle dependency and a single behaviour. Designing extensibility into each dependency and behaviour is a very bad idea, because it would make the design extremely complex. The open-closed principle should be applied in situations where the need for extensibility is either established or very likely. The “O” in SOLID must not be applied indiscriminately..

Contemplate the following quote of former US secretary of defense, Donald Rumsfeld:

There are known knowns. There are things we know that we know. There are known unknowns. That is to say, there are things that we now know we don’t know.. But there are also unknown unknowns. There are things we do not know we don’t know.

At first, this statement sounds a bit perplexing, but it makes perfect sense. It can be applied to our OOP problem as follows:

The known knowns are the features described in the functional and technical specifications. They are spelled out unambiguously and can therefore be translated directly into concrete implementations. The known unknowns are the features that we can predict, but their concrete implementation is unknown. This is where the open-closed principle applies. For example, if we write a printer spooler, we know that the spooler should work with all sorts of printers, although we don’t know the concrete printer models. Therefore, we provide an interface that can be implemented for any concrete printer model which he spooler then uses to perform its function. Finally, the unknown unknowns are the sort of features that we cannot predict at all.

As mentioned, designing extensibility for unknown unknowns results in unneeded complexity. The same is true for the known knowns since they are fixed. There is no need to design an interface for a class if we already know that there will be only one concrete implementation. Hence, the open-closed principle makes sense only for the known unknowns, namely the situations where feature extensions can be predicted with some degree of certainty.

February – 2nd Agile Principle

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

The second agile principle can be expressed in two words: embrace change. It is an appeal in opposition to the traditional fixed cost waterfall model. The waterfall model makes change as hard as possible. Every time a requirement changes, the waterfall model prescribes that specifications are rewritten, the estimate is updated and the contract renegotiated. As this involves considerable administrative effort, there is a high barrier to change. This barrier is intentional, because change is considered detrimental. The waterfall model attempts to eliminate uncertainties and perceives change a risk to project success.

However, if you have worked in project management for any length of time, you will know that change is inevitable. Organisations and businesses are not static. They grow and evolve. Their external environment puts them under constant pressure. Markets, competitors, changing regulations, and new technologies force organisations to respond and adapt. Likewise, internal changes happen all the time. When an organisation expands or takes on new projects, for example, internal structures and procedures often require overhaul. This in turn means that software requirements change. Because the degree to which organisations are computerised is increasing, the rate of software requirements change is also increasing. Consequently, we can expect that change is a constant and that the pace of change accelerates.

This is were the second agile principle comes in. The agile philosophy accepts change and uncertainty as “facts of life” so to speak instead of creating an artificial bulwark against it. The very meaning of agility is defined by this principle and this is a major departure from traditional project management. It is important to understand the implications of embracing change. What does this mean for the software development lifecycle and how does it affect the planning process? One can’t deny that welcoming change implies welcoming uncertainty, because change is hardly ever predictable. The farther one tries to look into the future, the less predictable it becomes. But how do you welcome uncertainty in project planning? The agile answer to this question is temporal focus.

In agile frameworks such as Kanban and Scrum, project goals are well defined only for the present and for the immediate future. For example, Kanban is like a pipeline where the focal point is narrowed on the tasks that are currently flowing through it. These tasks are described in detail, whereas the tasks that are queued receive less attention. Scrum is an iterative framework where the focus is on the current iteration (Sprint), whereas the backlog is out of focus. This means that tasks in the intermediate future are less defined. The degree of detail is limited and the time and effort estimations are rough. Tasks in the farther future are even more fuzzy. There may be large scale goals, roadmaps and ballpark estimates, but these are open to change. So, an important difference to the waterfall model is that projects are not determined from beginning to the end.

The fuzziness of long-term and indefinite future projects is not necessarily a disadvantage. It is the price of agility, so to speak. It affords nimbleness and the ability to harness change, which brings us back to the phrasing of the second agile principle. What exactly does it mean to “harness” change? Wasn’t change something that traditional project management tries to avoid? At the basic level, harnessing change means not being overwhelmed or overrun by external events. This is something that happened all too often in the past. Because agile methods are lightweight by definition, the agile team can respond and adapt to external changes quicker and more gracefully, even if such changes entail undesirable aspects.

At a more advanced level, change is proactive and initiated by the project team. For example, the project team might decide to adopt a new technology, or it may decide to include new features, or it may respond to a specific competitive challenge. The possibilities are endless. Because the cost of rethinking and overturning things is low in comparison with traditional project management, agile methods excel in responsiveness. This is especially relevant in environments that develop at a fast pace, such as emerging markets and technologies. Which brings us to the last term of the second principle: the customer’s competitive advantage. A competitive advantage is created when an organisation gets ahead of its competitors. Being the first to release a new feature or a new technology, for example, creates a competitive advantage. Agile project management is uniquely suited to such achievements.