The code is the documentation

The first time I heard someone saying: “the code is the documentation”, I thought it sounded completely wrong, like a lazy excuse for not producing documentation. However, it kept me thinking and I realised that there is also truth in this statement. This paradoxical thought-provoking quality makes it a proper mantra for agile practitioners, because it expresses a fundamental agile value: “working software over comprehensive documentation”.

Before we go further into details, I should dismiss the notion that agile developers don’t write documentation or misconceive its worth. Nope. We still produce documentation. However, we also apply the following agile principle to it: “Simplicity, the art of maximising the amount of work not done, is essential.” It is expedient to minimise documentation by adhering to practices that reduce the need for it. In agile development, documentation takes on the role of accessory parts whereas the primary attention is given to the codebase. Or more succinctly:

“Truth can only be found in one place: the code.” (Robert C. Martin, Clean Code)

The codebase is the ultimate source of truth. It is referenced in case of doubt, when a question is either too detailed or if the documentation is out-of-date. The code provides insight where no other method of reference is available. With this in mind, it is evident that code should be written in a way that is well-structured and understandable. Self-documenting code reduces or disposes the need for external documentation. Knowledge is represented in a single artefact and there is no need to synchronise multiple sources. Unfortunately, most real-world codebases do not have these ideal qualities. There could be many reasons for that, but the most common reason is that software entropy has taken its toll over time, and that too little attention was paid to refactoring.

So, code quality, self-documenting code, and continuous refactoring go hand in hand. It is important to understand that the highest code quality is achieved through conceptual clarity. Conceptual clarity comes from good naming and  good structure. This is by far more important than coding style, naming conventions, formatting and other external features, although the latter do of course contribute to code quality. Naming and structure cannot be tested automatically. Unlike code style, conventions, and formatting, they require human perception and intelligence. Which is one more reason to adopt such practices as code reviews and pair programming.

Coming back to “the code is the documentation”, I think the best way to understand this phrase is as an abstract ideal that ought to be worked towards. In an ideal world, code doesn’t need additional documentation, because it is so beautifully clear that it can be understood by anyone without any prior knowledge. It answers all questions that might arise about the software. It clarifies the intentions of the programmer and it is therefore also easy to maintain and change. Obviously, this is really difficult to achieve in the real world, especially across a large codebase, but its self-documenting properties are quite likely the best measure of code quality.

Code Review Antipatterns

Don’t you love code reviews? Having your grand solution scrutinised, criticised, and optimised by your colleagues makes you feel awkward? It doesn’t have to be that way. Code reviews are actually fun if done right and if the participants are aware of the pitfalls. They are not just conducive to achieving a high level of code quality, but they are also an ongoing learning experience.

There are three basic forms of code reviews: informal reviews (“hey, can you have a look at this?”), formal reviews (a formal task assigned to one ore more peers), or pair programming. The latter form has the advantage that it is done at the time when code is created. It is also the only form where two individuals engage in a direct conversation. There is no time lag. This makes pair programming the preferential form of review for high-priority tasks.

Agile teams working with git (or any other version control system) often encounter code reviews as a step in their code merging workflow. The typical unit of code being reviewed is a merge request, aka pull request in this setting. So, here are a number of things to avoid when doing a code review.

Not doing code reviews – Let’s begin with the obvious. Four eyes (or six or eight…) see more than two. Errors, problems, vulnerabilities, and bad design gets spotted early on. Corrective measures are still inexpensive at this stage. There is a dual benefit. Code quality increases and knowledge is shared. At least two people of a team are familiar with each feature or function, namely the author and the reviewer.

Confusing code reviews with functional reviews – Code reviews focus on code. Duh! The review is primarily (but not exclusively) concerned with the non-functional properties of the software. Functional reviews, on the other hand, ensure that functional requirements are met. The latter activity is closer to testing, therefore it makes sense to perform functional reviews in a separate procedure, probably as part of testing.

Bitching, nitpicking, fault-finding, patronising – It goes without saying that treating colleagues with kindness and respect is necessary for productive collaboration. Code reviews are no exception. Offensive behaviours only derail the process, while courteous and supportive comments go a long way. The reviewer must guard against coming across as a know-it-all and keep office politics out of the dialogue. In written reviews, annotators should aim at being helpful without being wordy.

Discussing code style conventions – Code reviews should not be about lexical code style, formatting and programming conventions. There are automated tools for this. If you care about formatting and code style, use a linter. More importantly, decide on a set of programming style conventions and configure the linter accordingly. It is totally OK if the team agrees to leave certain options open to individual preference. The formatting and style of code that passes the linter should not be argued, because it’s simply a waste of time.

Ignoring naming – Programming conventions may or may not encompass identifier naming conventions. However, linters cannot determine whether a given variable name is good or bad. Machines don’t care about names, only humans do. Phil Karlton has described naming as one of the two hard things in computer science. By all means, do review the identifier naming in the code review. As a rule of thumb, if an identifier name leaves you puzzled the first time you read it, it is probably not well chosen.

Debate – Code reviews aren’t meant as a podium for discussion. The objective is to improve the resulting artifact, not to find out who has the better arguments. Sometimes, people do have different opinions about practices and solutions. It may be helpful to resolve these by giving examples and references to commonly accepted best practices. If that is to no avail, a third party (or fourth or fifth…) should be be involved. In a written review, consider involving a third party, whenever the dialogue drifts into debate.

Not getting a third opinion – Involving a third party into the review is not only useful in cases where two people cannot agree. See first antipattern: four eyes are better than two, and six are better than four. If an important architectural question is at stake, why not involve the whole team? See: mob programming.

Reviewing huge chunks of code – Code reviews should be limited in scope, ideally to a few pages of code that can be completely read and understood in 20 minutes. Studies have shown that thoroughness and number of spotted issues are inversely proportional to the amount of code being reviewed. Therefore, results are improved if smaller units of code are being reviewed. Including time for reflection and annotations or discussion, the entire review should not take longer than 45 minutes. Anything exceeding one hour is questionable.

Unidirectional reviewing – The general idea is that everyone should review everyones code. Most teams have both senior and junior developers on board or perhaps a team lead or an architect. This often leads to the situation that seniors only review juniors, but not vice versa. This is wrong. First of all, seniors can make mistakes, too. Second, juniors can benefit from reading code written by senior developers. Third, varying fields of specialisation can be leveraged. A junior developer might have special knowledge or skills in a certain narrow area.

Rewriting code – It’s called code review, not code rewrite! Rewriting someone’s code is just not appropriate in most situations. It’s akin to pair programming where the navigator just grabs the keyboard from the driver whenever he feels like it. It’s simply rude. If the reviewer has a better idea, he should annotate the merge request with code examples.

Overdoing it – Programming is not a beauty contest. Code does not need to be perfect to be functional and maintainable. For example, even though I might prefer a functional loop construct over an imperative one, it’s just fine if the imperative one works. Perfectionism sometimes gets in the way and leads to other antipatterns such as premature optimisation and presumptive features (YAGNI). Also: A piece of code written for a one-time migration or ops procedure does not require the same high standards as production system code.

Underdoing it – Being too forgiving is likewise unhelpful. If the code review is executed as a mere approval formality, it’s a waste of time. Code should always be inspected for things like security holes, logical errors, code smells, technical debt and feature creep. It is important to strike a balance between clean maintainable code and reasonable effort.

Not reviewing tests – Unit tests are often mandatory and sometimes functional and acceptance tests are also added to the artifact. These should also be reviewed to ensure that the tests are present, logically correct and meet the standards. It’s a good idea to verify an agreed upon code coverage percentage as part of the review.

Not using a diff utility – Most teams probably use a tool set with built-in diff and annotation capability. In case you don’t, you make your work harder than it has to be.

1997

As the end of the year is drawing closer, it is time for some reflection. Let me take you back not just one year, but twenty. At the end of 1997, I found myself at the epicentre of the Asian financial crisis, also known as the Tom Yum Goong crisis, which originated in Thailand and followed the devaluation of the Thai Baht earlier that year. I was in charge of a startup company in Bangkok providing software services to local companies in Thailand. Within just a few months, almost all of our clients either became insolvent or withdrew from their contracts. As might be imagined, this presented quite a challenge for a fledgling startup business. At times, I was worried that I would not be able to pay out salaries to our employees at the end of the month. Fortunately, it never came to that. Due to providential circumstances that allowed me to bridge the worst periods with loans and advance payments, my company avoided joining the increasing numbers of Asian crisis casualties.

 

However, by the end of that year it became clear that the economic slump was of a more permanent nature and that I needed to restructure the company’s business if I wanted it to have a future. Fortunately, there were two things on our side. The Internet was taking over the world rapidly, creating a sudden demand for web services and web-savvy software and at the same time making remote collaboration more practical. Secondly, the company was small and agile, which meant it could adapt to these changes quickly. Thus I put all eggs in one basket and began to restructure the business as web development company. We moved away from the more traditional enterprise software market and embraced then emerging web technologies. I hired two graphic designers and our programmers learned HTML, Perl and JavaScript.

Perhaps more importantly, we started to look abroad for new business, particularly in Germany and the USA. The idea was not just to offer a new type of service, but also to offer it to a new type of clientèle, namely one that is located offshore in different places around the world. Within six months, between 80% and 90% of the company’s revenue was derived from web services, particularly from creating web sites for small and medium enterprises. As our portfolio was growing, we established a reputation and were able to attract bigger clients. By mid 1998 we merged with a small local web development company and thus cemented the path taken. The transition from working locally to working globally took a bit longer, however, as it turned out to be more challenging. Cooperating remotely with international clients presented not only technical and organisational difficulties. There are also a cultural barriers between Thailand, Europe and America that need to be taken into account.

Among the many cultural differences one finds besides language, communication style, business practices, social habits, the degree of risk acceptance, awareness of hierarchies just to name a few. Efficient communication turned out to be the most challenging issue. It became necessary to put project leads in charge who are not only fluent in English, but who also have some understanding of the way things are done outside of Thailand. In most cases, this adds a layer of management and administration. For example, requirement and specification documents have to be translated. Customer expectations must be communicated clearly to those who execute the work. By 1999, all of our major clients were international. It took almost two years to complete the transition from a local consulting business to an offshore services company. In the end, my company had escaped the snag of the Asian crisis. Perhaps more importantly, it had reinvented itself. The newly defined direction laid the foundation for years to come.

Web Framework Architecture

It appears that JavaScript web frameworks are repeating history by moving from a strict MVC push architecture to a push-pull component-based architecture. This is a trend that can be observed during the past two years. AngularJS is a case in point. The popular Angular SPA framework invented by Google has made a complete turnaround from version 1.4 to 2.0 and later. AngularJS 1.x and 2/4/5 are actually as distinct as frameworks of different vendors. Suffice it to say that the 2+ versions are not backward-compatible. It also seems that the current adoption of component-based JavaScript frameworks, such as Vue.js outpaces more conventional MVC architectures. Examples are Backbone.js and Knockout.js which are based on model-view-presenter (MVP) and model-view-viewmodel (MVVM) architectures respectively, both of which are elaborations of MVC.

But what is meant by repeating history? A similar thing happened in the Java world roughly 8-10 years ago, when first-generation frameworks like Struts 1 and JSF were replaced by newer frameworks such as Spring and Play that decoupled the request from the view/controller layers and introduced a bunch of new concepts such as dependency injection, domain driven modeling, and aspect-oriented programming, just to name a few. The same development could be then be observed in the PHP world, which inevitably follows Java trends with some delay. Case in point is the PHP Symfony framework that switched from an action-based push architecture to a component-based architecture in version 2.

To be frank, why JavaScript programmers have ever attempted to create frameworks with an MVC push architecture is beyond me. The whole approach seems unnatural, since JavaScript is native to the browser, and therefore native to the view layer. A push architecture implies that state changes happen on the server side, the effects of which are then computed and filtered and finally pushed to the view layer. It is easy to see that such a work-flow is more suitable to a Web 1.0 server application than to a JavaScript application. Of course, two-way binding resolves the dilemma partly, but it fails in decoupling controller logic and model changes from individual requests. The resulting problem can be solved by repeating logic across the application (bad!) or by introducing an application service layer (better but still unsatisfactory).

The natural starting point for a single page application is the user interface. The obvious thing to do is to structure the application around UI components based on an event-driven design similar to that of a desktop GUI application. Unfortunately, this is a bad fit for the request/response model dictated by classical web application architecture. A common approach is to decouple client state from server state and to marshal data back and forth through some low overhead protocol like JSON over HTTP/REST. It’s popular because it shields the application from networking concerns such as dealing with failing connections. However, a full-duplex TCP communication using a Websocket, for example, is a much better fit for such an application. As of now, not many web frameworks support using Websockets in a structured way.

There are, however, other areas in which JavaScript web frameworks are currently maturing and becoming more usable, once again following the example of existing server-side language frameworks such as Rails, Symfony, etc. by offering CLI-based scaffolding, support for validation, caching, ORM, DB migrations and whatnot. Possibly as an effect of server-side JavaScript, new build tools, and the growing JavaScript ecosystem, it is getting ever more viable to write large enterprise applications in JavaScript. Quite possibly, the most interesting developments of the upcoming year in web application architecture will happen in the JavaScript world.

A brief history of JavaScript

JavaScript is one of the most widespread and most successful programming languages today. Its history is reminiscent of a “from rags to riches” saga. It is said that JavaScript was created in only ten days. I am not sure about the historical accuracy of this claim, but suffice to say that JavaScript was born at a pivotal moment in mid 1995 when the Netscape Navigator browser was driving the commercial breakthrough of the Internet. Things happened fast at the time. JavaScript was conceived as a “glue language” for the browser and the document object model to give programmers a means of making web pages more dynamic. As for glue languages, most of them die when the product into which they are embedded reaches end-of-life. The browser, however, never reached end-of-life, but was continuously improved, and thus JavaScript kept growing and growing.

Its inventor, Brendan Eich, was tasked with creating a Scheme-like language targeted at amateur programmers for the Netscape browser. However, Java happened to be the hype of day and Netscape happened to be in a partnership with Sun Microsystems, the then owner of Java. Therefore, the language was christened JavaScript instead of LiveScript/Mocha and the syntax was made to resemble that of Java. Well, at least somewhat. To that end, the parentheses were dropped and the functional aspects of Scheme were retained. Surprisingly, a prototypal object model was chosen, slightly exotic at the time, possibly to allow easy object creation and extension without the need for complicated OOP constructs. It turned out to be a winning mix, because it allowed people to use procedural, functional, and object-oriented programming styles “al gusto”. Furthermore, one could use objects without even knowing anything about OOP.

Just a year and a half later, at a time when Netscape held over 60% of the browser market, JavaScript was submitted to ECMA International association for standardization and for overseeing the future development of the language. The first ECMA-262 standard was published in June 1997 and since then, JavaScript was also known as ECMAScript of which there are now several competing implementations. Strategically, this turned out to be a brilliant move. It ensured the uncoupling of JavaScript from the Netscape product. Eventually, this meant that JavaScript survived the browser wars, whereas Netscape Navigator did not. Around the same time, arch rival Microsoft created a clone language called JScript, a JavaScript “dialect” with small differences in its implementation. Though imitation can be seen as the sincerest form of flattery, this rang in a long-lasting period of browser incompatibilities, aggravated by Microsoft’s quirky DOM and CSS implementations, that caused web programmers a permanent headache.

The triumph of JavaScript continued, however. Netscape had the bright idea to turn JavaScript into a server language from the very beginning, but for some reason the world wasn’t ready for that in the 90ties. Just before the end of the millennium, the third edition of ECMAScript was released. It added regular expressions, exception handling, and a bunch of other things that made JavaScript a (somewhat) grown-up language. ECMAScript 3 remained the standard for 10 years and it gained significant leverage through Web 2.0 and Ajax. This technology allows data and partial web pages to be loaded from the server in the background through asynchronous application design. JavaScript is well-suited to asynchronous programming due to its functional nature. Since it is already present in the browser, JavaScript+Ajax enabled moving view logic entirely to the browser thus making applications more responsive. Two ingredients played a key role in this development: JSON and the jQuery library. A typical Web 2.0 application mixes HTML with JavaScript display logic and thus allows the creation of dynamic components handling data and visual content.

A natural outcome of this was the inclusion of library support for JSON parsing into ECMAScript 5 released in 2009. Because the complexity of JavaScript code increased with Web 2.0 innovations, “strict mode” was introduced in the same release. If enabled, it disallows the use of error-prone syntax and constructs, a kill switch for the weak sides of the JavaScript language, so to speak. An important event in the JavaScript world was the introduction of Node.js in the same year, a server-side JavaScript runtime environment based on Google’s V8 engine. Node.js allows the creation of server applications with non-blocking IO using an asynchronous reactive programming model. Node.js gained significant traction in the 2010s and is currently making inroads into enterprise server applications where it replaces Java and .NET applications. Another JavaScript driver is the single page application (SPA) model that redraws the screen of a single web page, akin to a desktop application, instead of successively loading web pages. Such applications are commonly implemented using a JavaScript web framework similar to server language web frameworks.

 

ECMAScript 6 (ES6), released in mid 2015, introduced a substantial number of new features and syntax to allow the creation of complex applications. They include block scope, arrow functions, enhanced parameter handling, modules, classes, enhanced object properties, assignment destructuring, iterators, promises, generators and new data types. Some of these features address the special needs of complex asynchronous server-side applications. Many developers perceive ES6 as the most important language update thus far. The ES7 standard published in the following year was a minor upgrade, adding an exponentiation operator and one additional array method. The ES8 standard released in June 2017 is the most recent major upgrade that ties in with the need for more complex language features for building server-side enterprise applications. It adds the async/await syntax which further simplifies asynchronous programming. It also provides new data structures and syntax to support concurrent programming, which opens the door to multi-threaded JavaScript environments.

JavaScript has turned from an ugly scripting duckling into a versatile and feature-rich full-stack web programming language. As of today, it is the web language most in demand and on its way to take over enterprise applications. It’s been quite a journey.