1997

As the end of the year is drawing closer, it is time for some reflection. Let me take you back not just one year, but twenty. At the end of 1997, I found myself at the epicentre of the Asian financial crisis, also known as the Tom Yum Goong crisis, which originated in Thailand and followed the devaluation of the Thai Baht earlier that year. I was in charge of a startup company in Bangkok providing software services to local companies in Thailand. Within just a few months, almost all of our clients either became insolvent or withdrew from their contracts. As might be imagined, this presented quite a challenge for a fledgling startup business. At times, I was worried that I would not be able to pay out salaries to our employees at the end of the month. Fortunately, it never came to that. Due to providential circumstances that allowed me to bridge the worst periods with loans and advance payments, my company avoided joining the increasing numbers of Asian crisis casualties.

 

However, by the end of that year it became clear that the economic slump was of a more permanent nature and that I needed to restructure the company’s business if I wanted it to have a future. Fortunately, there were two things on our side. The Internet was taking over the world rapidly, creating a sudden demand for web services and web-savvy software and at the same time making remote collaboration more practical. Secondly, the company was small and agile, which meant it could adapt to these changes quickly. Thus I put all eggs in one basket and began to restructure the business as web development company. We moved away from the more traditional enterprise software market and embraced then emerging web technologies. I hired two graphic designers and our programmers learned HTML, Perl and JavaScript.

Perhaps more importantly, we started to look abroad for new business, particularly in Germany and the USA. The idea was not just to offer a new type of service, but also to offer it to a new type of clientèle, namely one that is located offshore in different places around the world. Within six months, between 80% and 90% of the company’s revenue was derived from web services, particularly from creating web sites for small and medium enterprises. As our portfolio was growing, we established a reputation and were able to attract bigger clients. By mid 1998 we merged with a small local web development company and thus cemented the path taken. The transition from working locally to working globally took a bit longer, however, as it turned out to be more challenging. Cooperating remotely with international clients presented not only technical and organisational difficulties. There are also a cultural barriers between Thailand, Europe and America that need to be taken into account.

Among the many cultural differences one finds besides language, communication style, business practices, social habits, the degree of risk acceptance, awareness of hierarchies just to name a few. Efficient communication turned out to be the most challenging issue. It became necessary to put project leads in charge who are not only fluent in English, but who also have some understanding of the way things are done outside of Thailand. In most cases, this adds a layer of management and administration. For example, requirement and specification documents have to be translated. Customer expectations must be communicated clearly to those who execute the work. By 1999, all of our major clients were international. It took almost two years to complete the transition from a local consulting business to an offshore services company. In the end, my company had escaped the snag of the Asian crisis. Perhaps more importantly, it had reinvented itself. The newly defined direction laid the foundation for years to come.

Web Framework Architecture

It appears that JavaScript web frameworks are repeating history by moving from a strict MVC push architecture to a push-pull component-based architecture. This is a trend that can be observed during the past two years. AngularJS is a case in point. The popular Angular SPA framework invented by Google has made a complete turnaround from version 1.4 to 2.0 and later. AngularJS 1.x and 2/4/5 are actually as distinct as frameworks of different vendors. Suffice it to say that the 2+ versions are not backward-compatible. It also seems that the current adoption of component-based JavaScript frameworks, such as Vue.js outpaces more conventional MVC architectures. Examples are Backbone.js and Knockout.js which are based on model-view-presenter (MVP) and model-view-viewmodel (MVVM) architectures respectively, both of which are elaborations of MVC.

But what is meant by repeating history? A similar thing happened in the Java world roughly 8-10 years ago, when first-generation frameworks like Struts 1 and JSF were replaced by newer frameworks such as Spring and Play that decoupled the request from the view/controller layers and introduced a bunch of new concepts such as dependency injection, domain driven modeling, and aspect-oriented programming, just to name a few. The same development could be then be observed in the PHP world, which inevitably follows Java trends with some delay. Case in point is the PHP Symfony framework that switched from an action-based push architecture to a component-based architecture in version 2.

To be frank, why JavaScript programmers have ever attempted to create frameworks with an MVC push architecture is beyond me. The whole approach seems unnatural, since JavaScript is native to the browser, and therefore native to the view layer. A push architecture implies that state changes happen on the server side, the effects of which are then computed and filtered and finally pushed to the view layer. It is easy to see that such a work-flow is more suitable to a Web 1.0 server application than to a JavaScript application. Of course, two-way binding resolves the dilemma partly, but it fails in decoupling controller logic and model changes from individual requests. The resulting problem can be solved by repeating logic across the application (bad!) or by introducing an application service layer (better but still unsatisfactory).

The natural starting point for a single page application is the user interface. The obvious thing to do is to structure the application around UI components based on an event-driven design similar to that of a desktop GUI application. Unfortunately, this is a bad fit for the request/response model dictated by classical web application architecture. A common approach is to decouple client state from server state and to marshal data back and forth through some low overhead protocol like JSON over HTTP/REST. It’s popular because it shields the application from networking concerns such as dealing with failing connections. However, a full-duplex TCP communication using a Websocket, for example, is a much better fit for such an application. As of now, not many web frameworks support using Websockets in a structured way.

There are, however, other areas in which JavaScript web frameworks are currently maturing and becoming more usable, once again following the example of existing server-side language frameworks such as Rails, Symfony, etc. by offering CLI-based scaffolding, support for validation, caching, ORM, DB migrations and whatnot. Possibly as an effect of server-side JavaScript, new build tools, and the growing JavaScript ecosystem, it is getting ever more viable to write large enterprise applications in JavaScript. Quite possibly, the most interesting developments of the upcoming year in web application architecture will happen in the JavaScript world.

A brief history of JavaScript

JavaScript is one of the most widespread and most successful programming languages today. Its history is reminiscent of a “from rags to riches” saga. It is said that JavaScript was created in only ten days. I am not sure about the historical accuracy of this claim, but suffice to say that JavaScript was born at a pivotal moment in mid 1995 when the Netscape Navigator browser was driving the commercial breakthrough of the Internet. Things happened fast at the time. JavaScript was conceived as a “glue language” for the browser and the document object model to give programmers a means of making web pages more dynamic. As for glue languages, most of them die when the product into which they are embedded reaches end-of-life. The browser, however, never reached end-of-life, but was continuously improved, and thus JavaScript kept growing and growing.

Its inventor, Brendan Eich, was tasked with creating a Scheme-like language targeted at amateur programmers for the Netscape browser. However, Java happened to be the hype of day and Netscape happened to be in a partnership with Sun Microsystems, the then owner of Java. Therefore, the language was christened JavaScript instead of LiveScript/Mocha and the syntax was made to resemble that of Java. Well, at least somewhat. To that end, the parentheses were dropped and the functional aspects of Scheme were retained. Surprisingly, a prototypal object model was chosen, slightly exotic at the time, possibly to allow easy object creation and extension without the need for complicated OOP constructs. It turned out to be a winning mix, because it allowed people to use procedural, functional, and object-oriented programming styles “al gusto”. Furthermore, one could use objects without even knowing anything about OOP.

Just a year and a half later, at a time when Netscape held over 60% of the browser market, JavaScript was submitted to ECMA International association for standardization and for overseeing the future development of the language. The first ECMA-262 standard was published in June 1997 and since then, JavaScript was also known as ECMAScript of which there are now several competing implementations. Strategically, this turned out to be a brilliant move. It ensured the uncoupling of JavaScript from the Netscape product. Eventually, this meant that JavaScript survived the browser wars, whereas Netscape Navigator did not. Around the same time, arch rival Microsoft created a clone language called JScript, a JavaScript “dialect” with small differences in its implementation. Though imitation can be seen as the sincerest form of flattery, this rang in a long-lasting period of browser incompatibilities, aggravated by Microsoft’s quirky DOM and CSS implementations, that caused web programmers a permanent headache.

The triumph of JavaScript continued, however. Netscape had the bright idea to turn JavaScript into a server language from the very beginning, but for some reason the world wasn’t ready for that in the 90ties. Just before the end of the millennium, the third edition of ECMAScript was released. It added regular expressions, exception handling, and a bunch of other things that made JavaScript a (somewhat) grown-up language. ECMAScript 3 remained the standard for 10 years and it gained significant leverage through Web 2.0 and Ajax. This technology allows data and partial web pages to be loaded from the server in the background through asynchronous application design. JavaScript is well-suited to asynchronous programming due to its functional nature. Since it is already present in the browser, JavaScript+Ajax enabled moving view logic entirely to the browser thus making applications more responsive. Two ingredients played a key role in this development: JSON and the jQuery library. A typical Web 2.0 application mixes HTML with JavaScript display logic and thus allows the creation of dynamic components handling data and visual content.

A natural outcome of this was the inclusion of library support for JSON parsing into ECMAScript 5 released in 2009. Because the complexity of JavaScript code increased with Web 2.0 innovations, “strict mode” was introduced in the same release. If enabled, it disallows the use of error-prone syntax and constructs, a kill switch for the weak sides of the JavaScript language, so to speak. An important event in the JavaScript world was the introduction of Node.js in the same year, a server-side JavaScript runtime environment based on Google’s V8 engine. Node.js allows the creation of server applications with non-blocking IO using an asynchronous reactive programming model. Node.js gained significant traction in the 2010s and is currently making inroads into enterprise server applications where it replaces Java and .NET applications. Another JavaScript driver is the single page application (SPA) model that redraws the screen of a single web page, akin to a desktop application, instead of successively loading web pages. Such applications are commonly implemented using a JavaScript web framework similar to server language web frameworks.

 

ECMAScript 6 (ES6), released in mid 2015, introduced a substantial number of new features and syntax to allow the creation of complex applications. They include block scope, arrow functions, enhanced parameter handling, modules, classes, enhanced object properties, assignment destructuring, iterators, promises, generators and new data types. Some of these features address the special needs of complex asynchronous server-side applications. Many developers perceive ES6 as the most important language update thus far. The ES7 standard published in the following year was a minor upgrade, adding an exponentiation operator and one additional array method. The ES8 standard released in June 2017 is the most recent major upgrade that ties in with the need for more complex language features for building server-side enterprise applications. It adds the async/await syntax which further simplifies asynchronous programming. It also provides new data structures and syntax to support concurrent programming, which opens the door to multi-threaded JavaScript environments.

JavaScript has turned from an ugly scripting duckling into a versatile and feature-rich full-stack web programming language. As of today, it is the web language most in demand and on its way to take over enterprise applications. It’s been quite a journey.

 

The Vim Experiment

Though I started my career with text-based editors like vi, Emacs and Brief, I have been using IDEs for a very long time. It began with products with “Visual” in their name. Since then I moved on to Eclipse for Java programming, Netbeans for web/PHP development, Webstorm for Javascript and the list goes on. So far I have not looked back and never questioned the convenience and productivity that comes with contemporary IDEs. Until last month.

Someone suggested to give vim a try. Say what? 1970s technology instead of a full-featured IDE? Well, first of all vim must not be confused with vi. The latter is significantly older whereas vim was originally developed in the 1990s and is still in active development. Anyone who has ever worked with Linux is probably familiar with vim. It can be found on almost any *nix computer and often used for quick-and-dirty editing of configuration files. Perhaps it is not the most popular editor, because to the majority of people accustomed to non-modal editing, the modal interface of vim feels a bit foreign. In addition, vim has no point-and-click interface. It can only be used effectively by learning a great number of keyboard shortcuts and commands.

vim 8 on Ubuntu 16.04

So why vim? To put it simply, its curse is also its greatest promise. If your hands do not have to move between keyboard and mouse all the time, you can accomplish things faster and with greater ease. Drew Neil, the author of “Practical Vim” speaks of “editing at the speed of thought” and “high precision code editing”. There is also less potential for carpal tunnel syndrome with your hands resting on the keyboard. What is more, vim features a scripting language and a plugin system which makes it highly configurable and extensible. So the question is: can vim hold up a candle to modern IDEs or even beat them in terms of productivity?

I have decided to find out and prescribed myself a strict 3-month IDE-less diet using vim and nothing but vim for my daily editing work. Three months because, as mentioned, the learning curve is not exactly flat and it takes some time before all these keyboard sequences are committed to finger muscle memory. For me, there are two questions that I am looking to answer with this experiment. The first is whether vim can actually accomplish all the wonderful tasks that IDEs are good at and that make a programmer’s life easier, such as code completion, automatic formatting, diffing, syntax and code-style checking, debugging support and whatnot. So far, I am pleasantly surprised, though there are still a few rough edges.

The second question is whether typing speed and editing automation actually exceed the possibilities offered by an IDE and whether the promise of increased productivity does materialize. Not sure about this one either, although my vim repertoire is slowly improving and I start to feel like I am not merely hacking my way through the various editor modes anymore. At any rate, the vim editor is both ubiquitous and here to stay. So even if I decide to go back to using an IDE for coding, there is probably a benefit in mastering this tool a little bit better.

Object cloning in PHP

object cloningIn any complex object-oriented PHP program, there are situations that require copies of objects. Objects are often designed mutable, which means they contain state information that can change. Consider a bank account object, for example, which contains state information about balance, credit limit, and the account holder. Let’s assume that there are withdraw() and deposit() methods that change the state of this object. By contrast, an immutable design would require that withdraw() and deposit() return new account objects with updated balance information. This may sound like an irrelevant distinction, but the the implications are actually far-reaching, because mutable objects tend to increase complexity in subtle ways. Copying objects is a good example.

$object1 = new Account();
$object2 = $object1;

By assigning an object instance to a new variable, as above, one creates only a new reference and the object’s state information is shared by both reference variables. Sometimes, this is all a program needs. If withdraw() is called on $object2, both $object1->getBalance() and $object2->getBalance() return the same value. On other occasions, this behaviour is not desirable. For instance, consider displaying the results of a withdrawal operation on an ATM machine before the transaction is executed. In this case, we can make a copy of the account object, execute the withdrawal operation, and display the new balance or an overdraft message to the user without affecting the actual account. For this we need a copy of the object rather than a copy of the reference. PHP provides an intrinsic operation using the keyword clone:

$account = new Account();
$clonedAccount = clone $account;

The $clonedAccount variable contains a copy of the original object. We can now invoke  $clonedAccount->withdraw() to display the results and -with a bit of luck- the original $account object remains unaffected. With a bit of luck? Yes, unfortunately things aren’t quite straightforward. The clone operation creates a so-called shallow copy of  the original instance, which means that it constructs a new object with all fields duplicated. Any field that contains internal type data, such as integer, string, float, or an array is copied. If the balance is of type float, for example, we should be fine. If the balance field happens to be an object, however, we have another problem, because the clone operation does not copy composite objects but only their references. If the account class uses a balance object, a call to $clonedAccount->withdraw() method would still affect the state of the original $account object, which is clearly not the desired behaviour.

This can be remedied by adding a magic method named __clone() to the original object. The __clone() method defines what happens if the object is cloned:

class Account {

  protected $balance;

  function __clone() {
    $this->balance = clone $this->balance;
  }
  …
}

The somewhat odd looking syntax of the _clone() method above instructs PHP to make a copy of the balance object that the field $balance refers to when the object is cloned. Thus not only the account object itself is copied, but also the balance object that it contains. While this should be okay for our stated purposes, note that this only copies the balance object, and not any other composite objects that the account object might contain. It is not difficult to generalise the code, however. The following even odder looking syntax makes copies of all composite objects of the account object. It does so by iterating all fields of the current instance referred to by $this, whereas $key takes the names of the fields and $value their values:

class Account {

  function __clone()
  {
    foreach ($this as $key => $value) {
      if (is_object($value)) {
        $this->$key = clone $this->$key;
      }
    }
  }

}

The is_object() test in the above code is necessary to avoid cloning non-existing composite objects, i.e. fields whose value is set to null, which would result in an exception. Yet, this code still has a minor flaw. What if our object contains array fields whose values are objects? While the array itself would be copied, the array fields still contain references and thus would point to the same objects as the array fields in the original object. This flaw can be eliminated by adding a few more lines of code that make explicit copies of the array fields:

function __clone()
{
  foreach ($this as $key => $value) {
    if (is_object($value)) {
      $this->$key = clone $this->$key;
    }
    else if (is_array($value)) {
      $newArray = array();
      foreach ($value as $arrayKey => $arrayValue) {
        $newArray[$arrayKey] = is_object($arrayValue)? 
          clone $arrayValue : $arrayValue;
      }
      $this->$key = $newArray;
    }
  }
}

This already looks fairly complicated, but unfortunately it is not the end of our troubles. We also have to consider the hierarchical structure of composite objects, which means that the objects in object fields may contain object fields themselves which may yet contain objects with other object fields. Thus, creating a clone from scratch requires recursive copying of the object structure, otherwise known as making a “deep” copy. Obviously, the above method already gives us a way of implicit recursion if all of our objects implement it. The __clone() method of Object A is implicitly invoked by the __clone() method of object B when B containing A is cloned. We could use a base class for all of our objects to provide deep copying functionality. Although this works only with our own objects, and not with objects from third-party libraries, it would provide a comprehensive method for object copying. Unfortunately, the recursive approach still contains a flaw. Consider the following object structure:

class Employee {
  $name         = null; /** @var string employee name */
  $superior     = null; /** @var Employee employee's superior */
  $subordinates = null; /** @var array of Employee, subordinates */
}

This is an example of a class that represents a hierarchical graph of instances in memory. The Employee class defines a tree structure with the variable $superior containing a reference to the ancestor node and the variable $subordinates containing a reference to child nodes. Because of this double linking, the graph contains cycles, and because of these cycles, the above clone method will run into an infinite loop and cause a stack overflow. Cycles are fairly common in object graphs, though they are not necessarily as obvious as in the above example. In order to prevent the clone method from running into a cycle death trap, we need to add a cycle a detection algorithm, for example by reference counting. How exactly this is implemented is beyond the scope of this article. Let’s just say it’s not that trivial.

If you can do without cycle detection, there is a simple alternative for creating deep copies of an object, one which does not require a clone method implementation:

$object1 = new Account();
$object2 = unserialize(serialize($object1));

This takes advantage of the PHP serialize() and unserialize() library functions that  convert an object back and forth to a string expression. These functions take nested object structure into account. However, they are expensive operations, both in terms of CPU and memory, and they should therefore be used with discretion.