Ajax development with Dojo

Almost exactly 11 years ago I made my first steps into the world of web programming by writing a web-based address book in Perl. The idea then was to teach myself how to create dynamic web pages. It took me about a week to learn Perl and to write this small app. Last week I have repeated the exercise, albeit on a different level. I wanted to teach myself the fundamentals of the Dojo toolkit, Dojo Ajax, and some of the more advanced MySQL features. It was also a welcome break from a longer period of Java work. I used a PHP5 OOP framework for the server side scripting. Mind you, not one of the super-bloated third party frameworks that can do everything from spitting out Pi to the thousandth digit to calculating your mortgage amortisation, but my own small set of classes, which don’t do much more than providing a simplified MVC scheme, DB abstraction, session encapsulation, and authentication. The focus was on Dojo, of course.

Dojo Ajax Application

The good news first. Dojo is very, very powerful. Unless you do really esoteric JavaScript programming, Dojo probably fulfils all scripting needs you will ever have. It offers a great number of widgets from Delphi-like layout containers, windows, tabs, and menus to sophisticated tree controls, sortable tables, and data stores. The out-of-the-box widgets are very easy to create and use. Almost no scripting is required. All you need to do is to add Dojo custom attributes to HTML div tags. It is worth learning Dojo for the widget set alone. Widgets provide an easy way to enhance the GUI of web applications and to create powerful interfaces. But GUI widgets is not all that Dojo has to offer. The second most important thing is probably the Ajax facility, which is likewise a felicitous implementation of a complex functionality. Dojo offers several different transport mechanisms for Ajax requests, including XMLHttpRequest and IFrames. It hides all the underlying complexity from the programmer, such as mechanism differences and browser incompatibilities, and encapsulates Ajax requests into a single function call. Furthermore Dojo offers a JSON-RPC client for use with Ajax, drag-and-drop functionality, form validation, cryptology functions, graphic functions and animation, math functions, xml parsing and more.

Unfortunately there is also bad news. Dojo’s comprehensiveness entails complexity. Granted, this complexity is well hidden from the user. Creating a widget is as easy as adding some Dojo-specific attributes to HTML div tags and including the respective Dojo libraries into a JavaScript section in the head of the HTML file. Things start getting difficult when you interact with widgets and when you customise them. For most non-trivial applications you probably need to do both. What makes this task difficult is chiefly the lack of a proper API documentation. At this time, Dojo only has a partial documentation which means that you either have to look at sample implementations (of which there are only a few), or look at the Dojo source code, or scour the Internet for tips and hints. This can be quite time consuming and frustrating, as the specific information is not always available. It has been said that the Dojo documentation was even more fragmentary before version 0.4. Another problem I came across is that when I started to combine different widgets, such as splitters, containers and layout widgets, the Internet Explorer 6 could not render the page any more. I felt reminded of the bad old days of the browser war when cross-browser compatibility was a software engineer’s pipe dream.

Productivity was likewise not very exalted. I spent 60+ hours on developing a simple address book application (see screenshot), and I am not even counting in the time I spent on learning Dojo. This application has two screens, one for contacts and one for organisations. Each screen consists of an entity list that can be searched in two modes (simple search and advanced), a form with detail data of a single entity, and two linked tables. Almost all data exchange with the server occurs via Ajax calls. Each of the two pages is driven by 475 lines of JavaScript code that handle the dynamics, on top of Dojo. Although it was an instructive experience, and although I am quite impressed with the breadth and power of the Dojo toolkit, I did not find it very practical for real-world application development. Considering that a conventional PHP application could be produced in a third of the time, it would be more economic to use IFrames for producing a similar UI. I reckon that Dojo is still far removed from being a RAD tool for web development. Conceptually it seems to be on the right track, however. If a proper documentation becomes available and if widget integration and interaction is streamlined, it might come out as a winner. IBM and Sun have recently announced their official support for Dojo, so perhaps we should keep an eye on its future development.

Second thoughts about AJAX

At the end of 2006, it seems that the Ajax hype is finally cooling off. Perhaps this is a good time for a sober analysis. Ajax has indeed changed the Internet landscape, but has it really transformed the web into web 2.0, or has it been just a cosmetic face-lift? The jury is still out on that. In my opinion, Ajax is not just cosmetics, but cosmetic surgery. As such, it comes with the inherent dangers of the same. What is more, Ajax has failed to deliver on its web 2.0 promise. The real web 2.0 is a slow evolutionary process from an information delivery platform to an application delivery platform. Ajax looks more like an intermediate step to me. Currently we are in web 2.0 alpha stage at best.

However, there is one thing that Ajax has achieved. It has pointed out the shortcomings of the traditional web architecture quite plainly. Number one: state information has to be piggy-backed onto a stateless protocol; number two: the granularity of a page as the smallest unit of information is too crude for a web application; number three: web pages lack a mechanism for creating rich and sophisticated graphical user interfaces. Ajax addresses all three of these problems. It eliminates the need for state preservation by packing workflow into a self-contained page. It asynchronously loads small units of content and updates pages dynamically. It provides UI components and event models to the application by means of JavaScript libraries. So, what is bad about Ajax?

Ajax breaks the back button. Ajax doesn't preserve state. Ajax suffers from network latencies. These are the conventional objections, of which I have to say they are somewhat silly. The broken back button can certainly be fixed; besides it is a small cost for a greatly enhanced UI. State preservation reduces to the same problem. Network latency and response time constitute a much bigger problem for web applications that always reload entire pages, therefore it's hardly and argument against Ajax. So, again, what's bad about Ajax? From the user perspective – nothing really. Ajax does a lot to jazz up web applications and enhance user experience. From a software engineering point of view, however, Ajax is a real nightmare.

Web applications are already a complex mix of technologies, and Ajax adds even more complexity. The principal problem is that the code base is split into client side code and server side code and that they use incompatible languages. This naturally leads to code duplication, increased overhead, reduced maintainability and -in the worst case- Spaghetti code. As long as the client side code deals exclusively with display logic, the separation does not really constitute problem, except that you now have two different kinds of display logic. However, more serious problems loom when controller logic and business logic find its way into the client side code. The temptation for this to happen is just too great. Less experienced programmers and programmers working under pressure of time are especially susceptible. Sooner or later, you might find core logic added to the front end.

The second problem is that Ajax tries to solve an issue which it cannot really solve, namely that of creating a dynamic, rich GUI on a static, penurious platform. Granted, the Document Object Model allows for some dynamics, but the core problem is really given by the inherent limitation of HTML. The standard HTML elements just don't provide enough flexibility and extensibility for a rich GUI. The solution for this problem is not yet in sight, although several approaches exist, such as XForms, and proprietary user interface markup languages, such as XUL and XAML. Some of these solutions provide for Ajax-style refresh mechanisms, others are extensible, but none of them provides a solution to all GUI demands. Perhaps worse, there is currently no universal browser support for any of these languages. Ajax only appears to solve this problem, but in reality, it scripts its way around it.

The third problem is -quite plainly- JavaScript itself was well as browser-specific DOM implementations which make JavaScript/DOM code difficult to create and to work on a great variety of browsers. JavaScript defies or at least complicates standard development procedures, such as debugging, unit testing, integration testing, and automatic code generation. The same test procedures have to be run on different browsers. To cut down on development time, Ajax programmers often use libraries. Unfortunately, libraries tend to bloat web pages, because they provide fixed bundles of functionality, whether needed or not. Ultimately, a JavaScript programmer finds himself with a set of tools and procedures reminiscent to bronze age software standards.

Finally, developers should think hard about when to use Ajax and when not to. According to the KISS principle, throwing in Ajax just for the fun of it, is a bad idea. For example, if you have to display a hierarchical data structure with thousands of nodes, an Ajax tree component that can load nodes and branches dynamically, certainly comes in handy. If all you need is to display a DHTML menu and a few popup windows, you might not need Ajax at all. Using Ajax only for reloading partial page content is likewise a bad idea, in my view. Iframes offer a much cleaner and more maintainable solution to this particular problem. The tendency is of course that web applications become ever more complex and users become ever more demanding. Hence, it is inevitable that web GUIs and traditional GUIs will eventually converge. When this happens, Ajax will probably have served as a stepping stone to the final solution.

Ajax: a rising star

Until recently most people have associated the name Ajax either with a detergent or with a Dutch football team. This has changed as Ajax has caused a furore in the web development world. It began in 2005 with the introduction of new and highly interactive web applications, such as Gmail, Google Maps (maps.google.com) and Flickr (www.flickr.com), which are based on Ajax programming. Now Ajax is taking the world wide web by storm. The moniker Ajax stands for Asynchronous JavaScript and XML. Although often touted as new paradigm, neither JavaScript, nor XML, nor asynchronous data transfer is new. This is probably the greatest strength of Ajax.
Because Ajax makes use of well-known web technologies, and because the skill set for these technologies is common, Ajax is spreading fast. But we are proleptic. What exactly is Ajax and what does it do? Ajax is a programming paradigm for web applications. It utilizes a combination of four basic web programming techniques:

  • XHTML (in combination with CSS) for the user interface and web content.
  • JavaScript (or any ECMAScript compliant scripting language) in connection with DOM for the dynamic display of data and user interface components.
  • The XMLHttpRequest object (implemented in JavaScript) for the asynchronous exchange of information.
  • XML as a data format for data exchange (or alternatively plain text, JSON, or any other format).

The only thing new to web developers is probably the XMLHttpRequest object. It is the implementation of an API which can be used by client-side scripting languages to transfer data to and from the server in XML format. This API goes back as far as Internet Explorer 5.0 and the year 1999 when it sported the name “XMLHTTP ActiveX Object”. As such it was primarily known to Microsoft programmers and it led a relatively secluded life. Today most up-to-date browsers support the XMLHttpRequest object and recent web applications have exploited it in new ways to improve user experience.

So what does Ajax do? That is easy to explain. Let’s look at how traditional web applications work. You fill in some information, select some options, click a button and then the web server processes your request. During that time you wait. Then the browser renders a new page, you fill out some more information, select some more options and click another button. You wait again. This process is repeated over and over. At each step of the process the entire page has to be rendered anew. While you wait for the server response and the page refresh you cannot use the application.

Google CalendarAjax changes this. Let’s take Google calendar (www.google.com/calendar) for example. The Google calendar looks very much like a desktop GUI application. It features a monthly overview, a weekly timetable, an agenda, tabs, popup windows to display detail information, and so on. While you work with this calendar, say by retrieving the details of a certain event, the application connects to the server behind the scenes and retrieves event details from the database. Once the data becomes available to the client, it is immediately shown via DHTML without having to reload the page and without having to redraw the entire screen. While the data request is working in the background, you can still work with the calendar. Thus the application is always responsive and feels less clunky and slow than a traditional web application. In fact, it feels much more like a desktop GUI application.

Giving a web application the same look and feel as a GUI application and bringing it on par in terms of usability is -as it were- the Holy Grail of web programming. Until recently this has been very difficult to achieve. There are two reasons for this: the statelessness of web applications, and the lack of sophisticated widgets. The statelessness is a direct consequence of the HTTP protocol which does not deliver any context information to the browser, except for cookies and URL parameters. Hence, it is up to the web application to cache and retrieve session context information between successive page requests. The lack of widgets (or UI components) is due to HTML which is rather penurious with UI elements. There is a text field, a select box, checkbox, a radio button, a push button, and that is all you get. What is worse, the style and behaviour of these elements is difficult to control.

Does Ajax solve all these problems? Does it deliver on the web desktop GUI promise? Well, yes and no. Ajax provides great improvements in user experience by enabling asynchronous background processing through the XMLHttpRequest object. This functionality is great for filling data into UI elements and making a web application more responsive despite the transmission latency. It does not per se provide a richer user interface. The user interface still has to be coded manually and in case of Ajax this typically means DHTML code on basis of JavaScript, CSS, and DOM. A rich application with a variety of interactive elements, such as the mentioned Google calendar, consists of thousands of lines of DHTML code.

On the positive side, DHTML is portable. It runs on multiple browsers on multiple operating systems. It doesn’t require any plug-in or browser extension. This makes it a great choice over platform-dependent interface markup languages, such as XUL (pronounced “zool”) which runs only on Mozilla/Gecko browsers and XAML (pronounced “zammel”) which works only with Internet Explorer. The cross-platform compatibility of DHTML has to be taken with a pinch of salt, however. Since the ECMAScript and DOM implementations vary slightly from browser to browser, DHTML programs tend to be quirky and difficult to debug. At any rate they require rigorous testing. It is not unusual for DHTML programmers to spend more than 50% of their time with debugging.

One good thing about Ajax is that it reduces the amount of data transfer between server and client. It also reduces the web server’s CPU load considerably. This is why web service providers, such as Yahoo or Google, love it. Moving the processing load to the client-side effectively reduces their cost. It is certainly also an advantage in enterprise settings where a single machine or cluster serves a large user community. In short, Ajax is light on the server side and heavy on the client side, thus countering the well-known “thin client” approach with a “thick client” approach.

A further advantage which may come unexpected is that Ajax makes the implementation of an MVC (model/view/controller) architecture relatively simple. The client-side DHTML represents the presentation logic (V), whereas the server implements the model (M) and the controller (C). In practice, however, it is difficult to consign the entire controller code to the server, because this would result in an excessive number of requests and thus in considerable overhead. Depending on the nature of the application it may therefore be more practical to either move all application logic to the client, or to use a nested MVC model with controller modules on both sides. Either way, the MVC architecture is neither an intrinsic part of Ajax nor even a necessity, but it is certainly worthwhile considering when designing an Ajax application.

Web developers who have previously worked with a server-side scripting language, such as JSP, PHP, or ASP find that Ajax changes their world. Suddenly a huge chunk -if not all- of the application logic moves to JavaScript. The server-side scripts become lean and simple. In some cases they are reduced to a conduit to the backend, for example a SOAP server or a custom XML protocol. The ultimate Ajaxian approach is perhaps to rid the landscape entirely of XML and to use the JSON format instead. JSON represents data structures better to JavaScript. However, if the data is to be transformed into markup text, it may be more efficient to use an XSLT stylesheet to process XML and produce HTML output, rather than manually parsing and translating JSON.

So what are the drawbacks of Ajax? Are there any? Well, yes… We already mentioned its biggest disadvantage… JavaScript! Not that it is a bad language. Far from it. JavaScript is high-level, powerful, object-oriented, secure, and certainly very useful. However, it is more difficult to debug and maintain than pure server-side scripts. For example, with server-side scripts you never need to waste any thought on browser incompatibilities.

There are other disadvantages. One problem is that Ajax programs tend to mess up the functionality of the sweepingly popular and heavily used back button of the browser. The button doesn’t behave as users expect, because having eliminated successive page loads, the browser doesn’t keep a history anymore. There is a workaround for this. An invisible IFRAME element can be used for data transfer instead of the XMLHttpRequest object. The back button, or rather the history, remembers subsequent IFRAME page loads.

Are there any alternatives to Ajax? Yes, there are many alternative technologies which can accomplish the same as Ajax. Some are experimental, some are platform-dependent. There are two mature platform-independent products that allow the creation of rich GUIs and asynchronous data transfers, namely Java and Macromedia Flash/ActionScript.

Both of these products constitute interesting and commercially viable alternatives.
Of the two, Java is better known and more widely used. Java developers are probably surprised at the recent Ajax hype. After all, distributed computing is an integral part of Java. What Ajax does, Java programmers have been doing for years. The obvious solution for the delivery of GUI applications via the Web are Java applets. Unfortunately, applets are quite unpopular, because they are slow to load, isolated, and they require a plug-in. Other Java technologies, such as JSP/JSF or JSP/Struts, allow the creation of standard web applications with rich user interfaces. The downside is that they rely on a Java Server, or respectively a web server with Java-specific extensions.

The popular Adobe (formerly Macromedia) Flash is a client-side technology for the creation and delivery of vector graphics and animations. It comes with its own programming language named ActionScript which is ECMAScript-like. Thanks to ActionScript, the Flash product is capable of more than animation. A technique called Flash Remoting, i.e. RPC executed by ActionScript, accomplishes asynchronous data transfer using XML and AMF (ActionScript Message Format). The excellent graphic capabilities of Flash can be exploited to create rich web applications. However, there are some disadvantages. Flash is a proprietary single vendor technology; it requires a browser plug-in (the manufacturer claims that 95% of all Internet users have the Flash player installed), and it is graphic-centric rather than document-centric. It is still an excellent alternative to Ajax, especially for applications that make heavy use of graphics.