September - 9th agile principle
Continuous attention to technical excellence and good design enhances agility.
When reading this for the first time, one might think that technical excellence and good design are nice to have, but not essential. After all, a software project can be designed less than perfect but still function flawlessly and deliver value to its users. While this is true, there is a deep hidden problem. The culprit is perhaps the wording of this principle. The word “enhance” subtly implies a non-binding quality. Let me tell you from my experience that this assumption is way off the mark. In most real-life situations, sound design is absolutely critical. Likewise, striving for technical excellence is essential. The design and technical quality of a software project are the most important factors that determine its cost and longevity. The general idea is: design or die. A software system that is not well designed tends to accumulate technical debt quickly. It will therefore become hard to extend and maintain. It will also become costly and error-prone. So, unless one is working on small and simple projects, the ninth agile principle should be taken just as prescriptive as the others, if not more so.
To illustrate this case, let’s consider an all-too-common scenario which I call the “agile death trap”. Because agile teams work in short iterations with little upfront planning, solutions are always built towards current requirements. This can work well for quite some time. However, requirements do change over time. They tend to become more demanding. A feature that was built last year may not have been built with the current requirements in mind. Sometimes, new requirements necessitate changes to the underlying architecture of the software. Existing features need to be reimplemented. This is called refactoring, the process of restructuring software without changing existing functionality. Because refactoring adds no new functionality to the software, business cannot immediately derive value from it. Therefore business stakeholders tend to eschew refactoring. All it takes then to run into the agile death trap is a business stakeholder pushing for new features and a team member willing to apply a quick-and-dirty solution, either because they don’t know better, or because of a misguided “can do” attitude.
Let’s be clear. Sometimes, it’s OK to make technical compromises. After all, software development is driven by business goals and agile developers embrace change. Technically imperfect solutions are acceptable under one of two conditions: either as a temporary measure, or for single time use. Temporary solutions should be clearly delineated in the codebase. They should be replaced when appropriate. The agile death trap is much like a noose around the neck that tightens each time another technical compromise is made. The tightening of the noose eventually leads to the the scenario that was described in the first paragraph. This scenario is especially prevalent in teams where the balance between technical and business interests is noticeably tipped towards the latter. If business stakeholders don’t listen to the technical concerns of the team, or if developers are incapable of identifying, addressing, and eliminating such concerns, then the project is almost inevitably headed towards the agile death trap. Indicators of such a state of affairs are frequently missed sprint goals, decreasing code quality, high number of bug tickets, frequent release postponements, and frequently changing developers.
Ironically, teams that find themselves on their way into the agile death trap, often claim that they don’t have time for self-reflection and technical experiments. And they’re quite right, because they’re too busy with fixing bugs, incidents and operational hiccups. It’s a vicious cycle. The more time one spends with extinguishing fires, the less time is left for making the structure fireproof. And that is exactly the reason why attention to technical excellence and good design is not optional, but essential from the outset. This attention must be applied with discipline right from the beginning, because it is always cheaper and easier to fix technical problems early. Refactoring should not become an exception, but a habit that developers continuously apply while working on new features. If refactoring is applied continuously, the extent of refactoring work is likely to be small with each iteration. The team must communicate why continuous refactoring is necessary and how it improves code quality and the development process. Of course, work on non-functional attributes should just be as coordinated, or perhaps even more coordinated as work on functional attributes. It is often appropriate to create refactoring epics and stories, which in this case would be dev/ops stories instead of user stories, that pass through the same refinement and planning process as regular features.
But how does one achieve technical excellence? And what exactly constitutes good design? Let’s begin with the first question. This is frequently misunderstood. Technical excellence does not mean that teams must use the latest technologies. One does not achieve technical excellence simply by following the newest trends. Instead, technical excellence is achieved by using suitable technologies that are appropriate for the system’s requirements, understanding them well, and making optimal use of them. Technical excellence implies that the team is highly skilled with the tools, languages and technologies they use. It requires a focus on coding craftsmanship, best practices, engineering skills and professional teamwork. It is no surprise that professional teams often make use of a slew of practices with proven benefits, such as pair programming, peer reviews, coding guidelines, test driven development, automated acceptance tests, continuous integration, continuous deployment, and so forth. The choice of practices depends more on experience than on prescribed rules. Over time, a professional team finds out what works best for them. Technical excellence is not achieved in days or weeks. It takes ongoing dedication to learning, improvement and growth. It also takes dedication to producing high-quality software. This isn’t automatically given in teams that are composed of individuals with different backgrounds and ideas. It is something that requires hard work both as in cooperation as a team and individually.
Now about good design. Good design is not a luxury. It is an exigence for any non-trivial software project. It’s “design or die!” as illustrated in the death trap example. One must be aware that agile methods do not lead to good design in and of themselves. Quite the opposite actually, because agile methodology enforces emergent design. This means that the architecture and design of a software project evolve over time. We will discuss this in more detail in the 11th principle. Unfortunately, there is a lot that can go wrong with emergent design. Without proper diligence, emergent design easily spins out of control.
There’s a solution, however. Agile practice provides an iterative approach to emergent design. It is based on two main ideas: Just Enough Design Initially (JEDI) and continuous refactoring. JEDI dictates that the most simple design is chosen for implementing a given feature or epic. Continuous refactoring dictates that implementations must be refactored as soon as new requirements outgrow the initial design. The latter requires a periodic review of the current design. It also requires expert faculties of judgement, because identifying and correcting design weaknesses requires extensive training and experience. In a traditional waterfall project, architects are charged with this task. In a typical agile setting, design and architecture is left to the entire team. To make good design happen in an agile team, it is therefore important that team members have the required training and communicate well.
Which brings us back to technical excellence. It is almost always the case that the technical difficulty and engineering challenges increase over the lifetime of a large software project. Only a team dedicated to technical excellence, lifelong learning and continuous improvement can handle such a project well.