Lots has changed since I last posted about Fluid’s keyboard accessibility plugins for jQuery. Here’s a quick summary of what’s new:

We’ve donated our tabindex normalization functions to jQuery, and they’re available directly in jQuery 1.3. Here’s an example of how to use them:


$().attr("tabindex"); // Gets the tabindex of the first matched element.
$().attr("tabindex", -1); // Sets the tabindex value of all elements in this jQuery instance.


This now works consistently across all browsers in the jQuery way! There are some other tabindex-related conveniences in jQuery UI 1.7 core, too:


$().focusable(); // Returns true if the first matched element is currently focusable.
$().tabbable(); // Returns true if the first matched element can be focused using the Tab/Shift-Tab keys.


Thanks so much to Scott Gonzalez for helping us land these patches in jQuery!

In my previous blog post about keyboard accessibility, I hinted at a comprehensive keyboard accessibility library that makes it easy to support desktop-style navigation with the keyboard. This plugin now ships as part of our Fluid Infusion application framework. It also works as a standalone jQuery plugin if you don’t need the rest of Infusion’s great features.

If you’re interested in this plugin, here are a few links to get you started:

• Keyboard Accessibility API Documentation
• Keyboard Accessibility Tutorial
• An example showing keyboard navigation and ARIA with jQuery UI Tabs
• Source code for the jquery.keyboard-a11y.js plugin

As always, comments, feedback, and suggestions are welcome.

World, Meet Infusion 1.0

The Fluid community shipped Infusion 1.0 a couple of weeks ago. Before making a splash with announcements about the release, we wanted to take some time to revisit our documentation and sample code to be sure they are really useful and comprehensive. Recognizing that it’s not enough just to ship a bundle of code, we try to provide a whole package of resources for developers and designers who are using Infusion. That means comprehensive API pages, tutorials to get you started, and concepts documentation to give you insights into the philosophy and design patterns we’ve built into Infusion. With this new documentation in place, now’s the time to share the news of Infusion 1.0’s release more widely.

Fluid Infusion is an application framework for building usable and accessible user interfaces with JavaScript. Built on top of jQuery, Infusion takes a different approach to client-side development. At heart, Infusion is an open architecture designed to put you back in control of your application’s user experience. It includes a growing collection of UI components—reusable interactions that go deeper than most widgets. Created by a community of developers and interaction designers, Infusion components are built from the ground up with accessibility in mind. All of our designs can be used with assistive technologies, are fully controllable with the keyboard, and can be transformed to suit your users’ personal needs.

Knowing that you can’t bottle good design, our components can be easily adapted and reworked to suit your creativity. We give you the ability to tweak more than just a few stylesheets—you can totally change the markup of a component, change its behaviour, or toss your own workflow and logic into the mix.

What’s an Application Framework?

Most JavaScript libraries fit into two categories: foundation toolkits and application frameworks. Foundation toolkits are probably the most familiar, providing all the basic tools required to manipulate the DOM, manage events, and make Ajax calls in a reliable, cross-browser way. jQuery and Prototype are probably the best-known examples. Application frameworks, on the other hand, have emerged more recently. With more complex interactions and more application logic moving from the server to the client, there is a need for tools to help manage growing code bases. Application frameworks such as Sproutcore and Dojo + Dojox often provide features such as data binding, support for the Model-View-Controller pattern, and HTML template rendering.

Infusion puts a unique spin on things: it takes a declarative and event-driven approach, promoting less code and greater flexibility. Built on top of jQuery, everything about Infusion is open: its architecture, source code, and community. Our approach to Model-View-Controller is different, too. Models and views are at the heart of Infusion, while controller code is conspicuously absent. Unlike most MVC web frameworks, we think it’s the job of the framework to take care of the glue, leaving you to focus on the stuff that makes your application tick.

Infusion’s client-side template renderer is completely unobtrusive, allowing you to dynamically render your user interface without having to mix up bits of code with your HTML. No code in your templates means more reuse, less chance for mistakes, and an easier job when it comes time to do a redesign. We also include a lightweight CSS framework that lets you mix and match styles to create beautiful and themeable user interfaces, providing pre-baked layouts, widgets and text styles to get you started. We even give you plugins and framework APIs to help ease the burden of creating user interfaces with JavaScript that will be accessible to a wide range of users.

Interoperability is huge for Infusion. We play nice with other JavaScript code, avoiding the conflicts that are common when combining different JavaScript libraries. This makes Infusion particularly well suited to building mashups, portals, and content management systems.

Fluid and jQuery

We love jQuery. Infusion wouldn’t have been possible without the simplicity and clarity of jQuery’s approach. We also didn’t want to reinvent the wheel, so jQuery is everywhere in Infusion. Recognizing that some applications need more, we created Infusion to complement jQuery and other JavaScript libraries. It’s not an either/or situation—Infusion is very modular, so you can pick and choose the features you need based on the complexity of your project. Go ahead and combine ordinary jQuery code with Infusion, or use the whole framework whenever you need robust data binding, unobtrusive markup generation, and MVC-style designs.

Fluid’s culture of collaboration and interoperability run deep. To that end, we are active contributors to jQuery UI, sharing our accessibility experience and code to help make jQuery even better. Every time we come up with a new accessibility improvement, we share it back with jQuery so that the whole community can benefit.

What’s Next?

This is just the beginning. With Infusion 1.0 out the door, we’ve got lots of new features and improvements on deck. Indeed, the past few weeks have been packed with tweaks, bug fixes, and features destined for the Infusion 1.1 release. We’re planning to ship this next release in early June. Beyond that, we’ll continue to grow our components and add new features to the framework.

We’re also active users of Infusion, building applications and helping other open source communities with their user interfaces. To this end, the Fluid community is embarking on a new project called Fluid Engage, in collaboration with museums and art galleries. We’ll be working alongside curators, education departments, and exhibit designers to help create mobile-friendly open Web solutions that bring a new level of interactivity and visitor engagement to the museum. Infusion will be the backbone of this effort, and Engage will undoubtedly drive new components and framework features back into Infusion. Keep an eye out for things like new mobile themes for the Fluid Skinning System, designed to provide a natural user experience and to repurpose content for mobile devices like the iPhone. We’ll also be shipping a new visualization library for Infusion, providing Canvas-based solutions for drawing rich, vector-based images and animations. If any of these features interests you, don’t hesitate to drop us a line if you’d like to lend a hand.

Give it a Spin and Let Us Know

We’re really excited to share Infusion with everyone. With the 1.0 release, Infusion offers a stable and reliable solution for building rich user experiences, supported by a dedicated and friendly community. We’ve got a healthy roadmap moving forward, and a growing community of users. With accessibility and usability baked in from the start, we’ve got you covered. Download the release, check out the documentation, dive in, and let us know what you think!

In preparing for the final release of Fluid Infusion 0.6 (as of writing, we are at “Bug Parade” stage, leading to code freeze for Monday), we found ourselves with the disagreeable requirement to make some incompatible API changes to a part of the framework. Not only why we had to do this, but what we did about it, are interesting.

The code in question was kept in the file jquery-keyboard-a11y.js, and has historically always had a “special status”. This holds the parts of our framework which we have always considered closest to being contributed to the wider jQuery community (who we are always eager to develop productive relationships with), as a plugin. This code is treated specially, in that special care is made to keep it as self-contained as possible, and in particular to keep it free of dependence on our standard framework base file Fluid.js, to ease its adoption outside our community.

Unfortunately, as of release 1.5, the jQuery UI project included a file ui.selectabe.js, which registers a plugin named “selectable” with the same name as one in our set. This is interesting, since it highlights what has been considered a weakness in the jQuery plugin model - that it does not support any infrastructure for namespacing, to ensure that loosely cooperating teams can continue to work in parallel without fear of treading on each other’s names. This partly reflects its origins - the jQuery “chained filter” idiom is admirably matched for expressing queries and primitive operations on the DOM, and has produced a revolutionary condensation in the amount of code necessary to perform these “bread and butter” tasks. The plugin model is less well suited to an idiom of construction - that is, when one is using the DOM as a “springboard” for the construction of some more complex “component” or tree of objects that is going to spend some durable lifetime managing it. This can be seen from the oddity of some code which constructs say a dialog, or a sortable - where *is* the dialog in the following call?

 $("#dialog-root").dialog({bgiframe: true});

Any “object” actually referring to the dialog is implicit - the jQuery standard, is that operations on a jQuery object return another jQuery object (barring the exceptions that they are queries for primitive values such as attributes or counts) - therefore, to interact with this “dialog” we must continue to use same-named plugin calls targetted at the same DOM node (another interesting oddity is that the constructed object is not tied to the lifetime of the invoking jQuery object, but to the durability of the DOM nodes addressed by it - this is by necessity, since jQuery objects have no durability).

As a quick aside to those who have not been following these chain of blogs avidly, the Fluid framework design recommendation, since before the 0.5 release, was for “that-ism“, a code structuring device originally recommended by Douglas Crockford. I made an earlier blog posting explaining our decision, and there is also a wiki page on How to define a Unit. In brief, the benefits of that-ism include greater reliability and predictability of code - in the months since our decision we have been very happy with it, with many contributors commenting that it has made our codebase easier to understand by reading. That-ism also interacts nicely with Javascript’s natural “detached function” idiom - for example it is very easy to write a line such as

        that.viewEl.click(that.edit);

without having to worry about awkward “this” re-dispatching through event cycles etc - the edit method has natural access to its own, user-defined, and limited, context, without the need for “this juggling”. The durability of the returned that from a constructor is also matched to the created object - since it is the created object.

In any case - the jQuery model is marvellous for encoding primitive DOM operations drawn from a fixed set, slightly odd for construction, but poor for namespacing - having lost our namespace for selectable our only option is to move out of the path of the elephant. However, there is no procedure how we can guarantee that further of our names will not collide in the future, so now is an excellent opportunity to think of a scheme to make sure that our users will never again suffer this kind of API incompatibility.

Also, over the lifetime of Fluid, jQuery have firmed up their recommendations and published the following page of Guidelines for plugins - which we could do better at meeting. In particular, they recommend that each thing designated a “plugin” use exactly one name in the plugin namespace, in an attempt to head off the namespacing issues I’ve mentioned.

So, this problem is an excellent opportunity. Can we, at the same time, improve our conformance with jQuery code idioms, solve our namespacing issues permanently, and at the same time improve jquery-keyboard-a11y.js in its agreement with the that-ism enjoyed by the rest of the Fluid codebase?

It sounds impossible, but actually we can. After a lengthy brainstorming session, during which we proved that a “natural” namespacing idiom (that is, one which allows function names to be written out as Javascript symbols, rather than as string) is impossible to apply together with a “this-ist” dispatching model, we came up with the concept of the “that-ist bridge”. This is a small 20-line “machine” which now currently sits at the top of jquery-keyboard-a11y.js, and does the work of “exposing”, not only the contents of that file, but in fact the entire Fluid framework (where it has potentially conformant arguments) as a single, gigantic jQuery UI plugin named fluid, with exactly the recommended jQuery UI argument semantics.

Here is an example of our old syntax, showing invocation as a jQuery UI plugin named selectable:

$(".my-nodes").selectable({direction: $.a11y.orientation.HORIZONTAL});

“On the ground”, this has now been rewritten to a that-ist, Fluid-standard construction, used as follows:

fluid.selectable(".my-nodes", {direction: $.a11y.orientation.HORIZONTAL});

This is now housed in the global fluid namespace owned by us, permanently heading off the potential for collisions in the future. However, owing to the miracle of the “that-ist bridge” machine, this function is now also accessible, for free, using the jQuery UI plugin syntax as follows:

$(".my-nodes").fluid("selectable", {direction: $.a11y.orientation.HORIZONTAL});

Essentially any function, which accepts a “jQueryable” as its first argument (that is, anything which can legally construct a jQuery object when used as an argument to $, that is, selector strings, DOM nodes, or jQuery objects themselves) will be automatically adapted by the “that-ist Bridge” to be invocable using the single, global, Fluid jQuery UI plugin. The arguments are permuted - the first argument to the function becomes the jQuery object itself which the plugin is invoked on, and the symbol for the name of the function becomes a string, the first argument passed to the plugin. The remaining arguments to the function may be passed in an array as the 2nd plugin argument, or if there is a single argument it can be sent directly.

The treatment of return values is also somewhat interesting. The jQuery standard is that jQuery operations return further jQueries - except where they don’t. It seems that the general exceptions are those which return primitives - so this rule has been formalised in the bridge code. The return value of the underlying Fluid function is examined for its type. If it is a primitive, or “falsy” value, it is return directly. If it is some other kind of value, it is assumed that it is some kind of “Fluidic that”, and the value is hidden - instead, the original jQuery this is returned, to allow the standard jQuery selector chaining idiom to work. However, as a courtesy, the specific jQuery object returned is decorated with an extra method that() by which the originally returned value can be retrieved.

This has been a big win all round - since at the same time we have been able to improve conformance both with our own coding standards, as well as jQuery’s. And further, we have now been able to extend to the contents of this file the protection of our version management strategy, first brought in in Fluid 0.5 - by means of a namespacing trick, multiple versions of the Fluid framework may be included into the same document, whilst at the same time not only not colliding in namespace, but even remaining simultaneously addressible from the same scope by specially aware code. Truly necessity is the mother of invention. Or else, a loaf of bread.

There are many ways of finding things in the DOM - sometimes, many ways of evaluating apparently the same expression. For example, CSS and JQuery allow you to find elements by id, using an expression of the form

  var myElement=$("#my-element-id");

as well as the apparently equivalent expression

  var myElement=$("[id=my-element-id]");

However, these expressions are different in many ways. One advantage of the latter is that it is more resistant to escaping issues - for example, the colon character : is actually a valid part of an XML id, but is not correctly handled in many escaping situations. JQuery will not accept it as part of a # expression, but is fine in an id specification. This was the basis of our adoption of the latter for the fluid.jById utility.

However, these types of selectors are treated very differently within the nuts-and-bolds of the jQuery selector engine. # selectors have a very special status, which enables numerous shortcuts including the judicious use of document.getElementById where possible. id selectors are just standard attribute value filters, and will perform a full scan of the DOM.

These tests were done on Firefox 2 in a rather busy document of about 40K of HTML.

Note that the cost of simply wrapping a DOM element in a jQuery object is small, but still noticeable.

Cost of guarding

This immediately suggests a policy of falling back on document.getElementById for quick lookups - however, jQuery, being a mature framework, is doing some work for us here in insulating us from an awkward IE bug where document.getElementById is sometimes returning an element which has a match on *name* rather than a match on *id*. This is a rather awkward tradeoff - putting in a “guard branch” to check that the required id is actually present as an attribute slows down access for everyone to 22µs, with the option to default back to the jQuery scan. However, do we really want a library with the contract “give me the element with this id, unless there is something screwy about your document, in which case give me the element 1000 times slower”. Probably the user will not thank you in this case, and would prefer to be given the chance to fix their markup.

Quick access to DOM nodes

This brings us back to why any of these issues are particularly interesting in the first place. I am currently working on reforming the Fluid Reorderer component, not only from a structural but also a performance point of view. The behaviour of the jQuery droppable library is not quite what we would want, both from a performance as well as a behavioural point of view (might be the subject of a separate blog posting), but in fixing it we are faced with some crucial design decisions about how to arrange data structures such that they can be used for very quick access - ordinarily this is not so important, but a DnD library is going to be queried on every mouse move event, and so even timings in microseconds can begin to mount up.

There are two main options -

i) store a “reverse lookup” cache of jQuery.data id numbers onto DOM nodes, and work with data structures storing jQuery.data ids, or
ii) synthesise short unique ids onto DOM nodes that do not have them, and work with data structures storing DOM ids.

Route i) runs the risk of keeping these caches up to date, and in particular avoiding leaks due to “expired” DOM nodes continuing to hang around in the cache - but it would probably be faster at runtime. Route ii) will be a little slower (incurs the costs we have been talking about so far, in invoking a document.getElementById equivalent on going to the DOM) but is definitely leak-safe, and might also help others navigate around the DOM.

What to do

These are hard kinds of calls to make, and involve a consideration and weighting of different kinds of issues. Currently a reasonable tradeoff for fluid.jById seems to be to go towards document.getElementById, but to put in a guard branch with a hard failure if a “name” attribute is picked up by mistake. For “live” storage during drag-and-drop, the jQuery data id cache option appears preferable, since we will already be storing various other data on drop targets in addition to what can be discovered in the DOM.

jQuery.data is really pretty fast, I will leave you with a few more timings -

In the interests of aiming towards slightly more bite-sized blog postings, I thought I would share with you some possibly unsuspected gems in a favorite JQuery method I kicked over recently.

Basic extending

jQuery.extend is more commonly seen as part of plugin code - the jQuery plugin model simply works by “extension” of the jQuery object itself to include new methods. For example, here is some code from the JQuery UI dialog plugin:

$.extend($.ui.dialog.overlay, {
	instances: [],
	events: $.map('focus,mousedown,mouseup,keydown,keypress,click'.split(','),
		function(e) { return e + '.dialog-overlay'; }).join(' '),
...

The extend call is taking the existing jQuery.ui.dialog.overlay object (initially set up to be a {}) and “mixing in” a set of properties and functions from an “immediate hash”.

Multiple arguments

So far, this would account for the powers of a 3-line function, and one which we have all probably independently written for ourselves (certainly in the historical forms of fluid.mixin and RSF.assign). However, where jQuery.extend really shines is in its extensions. Firstly, it will accept any number of arguments - which are just progressively mixed in on top of its first argument:

$.extend(target, source1, source2, .... sourcen);

will “blat” all the members from the source arguments serially to arrive on top of target.

Deep copy

However, the real money of jQuery.extend is in its “deepCopy” functionality. Supplying the boolean value of true as the first argument will not just copy properties laid directly on the “source” arguments, but recursively copy any properties it finds. Here is some code from the new Fluid core initialiseThat function:

      that.options = jQuery.extend(true, {}, fluid.defaults(componentName), userOptions);

This has replaced countless quantities of “options” manipulating code from around the different components, some of it quite fiddly and manual, with a single, clear, static policy in one line. The core JQuery documentation page for jQuery.extend explains this option, but in a way which might easily be missed at a quick reading. As many of the example show, it is typically used for merging “options” structures, but it is also invaluable as a plain “deep copy” operation, for example forming the basis of Fluid’s upcoming pure model idiom. Tricks like this help keep down our core library footprint.

Note on Javascript types

In such a staggeringly weakly-typed language as Javascript, this kind of useful “overloading” is generally impossible. However Javascript is at least blessed with the ability to always distinguish a handful of primitive types (string, boolean, number) from everything else, and so enables useful “Swiss army knife” methods like jQuery.extend - luckily it detectably has no meaning to try to be extending a boolean object. One way to perform this type detection is by use of the typeof operator, JQuery itself chooses to use the following test in this situation:

  if ( target.constructor == Boolean ) {

Reading through the JQuery source base is highly recommended for people with a few moments to spare, who like to know what people who know what they are doing look like

This posting will be about the different styles of creating “components” or “objects” in Javascript, and which styles Fluid is planning to recommend and in what contexts.

Javascript has inherited an “infusion” of “object-oriented” features grafted at the last moment on its fairly minimal and clear (LISP-inspired) baseline. The resulting language simply has too many ways of doing the same thing, and numerous features that are downright dangerous and obscure.

Continuing our chats on the performance theme, Colin and I set out to try to find a simple set of recommendations that would lead to code that is not only safe, but readable, maintainable and performant. Particularly, the aim is to cater to requirements of more structured programming, by larger, looser teams sharing components over longer timescales, rather than the “one-man”, ephemeral programs for which Javascript is more obviously suited.

In this posting I will cover a full range of “componentisation” techniques available in Javascript — from simple namespacing, structural use of closures — “scope-ism”, blending into the new “that-ism” that Fluid will promote, and contrasted against the “traditional” object-oriented language features (”this-ism”) based on this and prototype that we will deprecate, as well as more old-fashioned but performant and clear styles (”arg-ism”) carried forward from “pre-object-oriented” languages.

Open and closed

Continue Reading »

Back in April, Eli Cochran and I organized a half-day workshop on JavaScript development techniques at the JA-SIG Spring conference in St. Paul, MN. Writing Fearless Javascript for Portlets, Widgets, and Portals was designed to give attendees a primer on various aspects of client-side user interface development, including JavaScript fundamentals, the use of toolkits such as jQuery, and DHTML accessibility.

The workshop explores what I’ve been referring to as “modern” JavaScript programming, for lack of a better term. In contrast to the previous generation of JavaScript programs, where small bits of interactivity were sprinkled on top of a largely static document, modern JavaScript involves the creation of richly interactive applications and components. These days, your code probably shares the page and Javascript’s notorious global namespace with other libraries and widgets. As a result, you need to do things a bit differently to ensure your code is self-contained and won’t conflict with other libraries. Fearless JavaScript gives you the techniques needed to thrive in the world of portals, plugins, and mash-ups.

We’ll be presenting a new edition of Fearless JavaScript at the Sakai Summer 2008 conference in Paris, France. The material has been significantly refined, and we’ve added another co-presenter: Nicolaas Matthijs from CARET at the University of Cambridge. Nico will be talking about how to create gadgets for Sakai using JSON-based data feeds. We’re working on some tutorial code that walks you through using Fluid components within a Sakai gadget. Cool stuff.

Nico attended the original Fearless JavaScript workshop in St. Paul, and had lots of interesting questions and suggestions for us. Afterwards, Nico and I spent an enjoyable couple of hours working together on some of his gadgets, tweaking their accessibility and talking about his designs. He’s doing some great work on MyCamTools, and I’m looking forward to sharing the stage with him and Eli for Fearless JavaScript 2.0.

If you’re in Paris for the Sakai conference, we’d love to see you at the workshop. If you can’t make it, we’re hoping to create a podcast of the event. Here are the details:

Fearless JavaScript
Presented by Colin Clark, Eli Cochran, and Nicolaas Matthijs
Monday, June 30
1:30 - 4:30 pm in Amphi 55A

Topics will include:

* Why JavaScript?
* Fundamentals of JavaScript
* Using jQuery
* JavaScript in Sakai
* Portal and mash-up friendliness
* Accessibility

Everyone is welcome to attend the workshop. It’s assumed that you have some understanding of at least one programming language, and that you’ll bring a laptop to follow along with the hands-on exercises.

See you there!

Following up on Antranig’s recent post about performance, I’d like to address the issue of code design and its relationship to performance. Antranig’s article is very comprehensive and well-considered, and it provides us with a number of guidelines for balancing the tradeoffs between code quality and speed. However, I was surprised that he didn’t remind us the C.A.R. Hoare quote made famous by Donald Knuth:

“Premature optimization is the root of all evil.”

Antranig’s performance metrics are intended to help establish a gut feel about the cost of a particular operation. They’re low-level measurements, and as such they can’t take into account a fundamental variable: context. These performance metrics have very different consequences depending on the size, scale, and nature of the application. His examples are apt: intensive, highly recursive, up-front operations like template rendering show the full cost of JavaScript’s weak performance. Indeed, Antranig has used his research to good effect on our nascent client-side template renderer. In a relatively short amount of time, he was able to improve the performance of his low-level parsing algorithms by several orders of magnitude using regular expressions and fast string comparisons. Impressive stuff.

On the other hand, context matters a lot when it comes to user-facing behaviour, where timings may vary significantly based on the user’s choices. Many UI components, such as the Uploader, provide a fairly modal experience for the user. Such components undoubtedly have less stringent performance requirements, and our acceptance criteria should be determined by testing in real-world situations.

The risk with out of context performance measurements is the tendency to optimize prematurely and without accurate data. There’s only so much time in the day. Don’t lose the opportunity to add useful features to your application by starting in on time-intensive performance optimizations too early. Without good data, your hard work may only have a tiny impact on the overall perception of speediness. Worse yet, premature optimization can lead towards architectures that are poorly factored, redundant, and hard to maintain.

So how do we ensure good performance without compromising architectural quality?

1. Write code with good design principles in mind
2. Continually measure performance in real world situations
3. Refine code incrementally to improve performance

Let’s take jQuery as an example. If you look at the first release of jQuery, you’ll find that it was mostly composed of small functions that are clear and easy to read. jQuery’s author, John Resig, has always done a very good job of carefully profiling his code with performance tests, and over time the code base has evolved in certain areas to be faster and, as a concequence, occasionally less readable. He took the time to build something real first, then he optimized strategically. This is exactly the example of how we should all handle performance optimizations: build it cleanly, then identify areas where performance optimizations are necessary. Optimization without real-world numbers is, well, evil.

Building on Antranig’s advice, here are some strategies to help ensure both good performance and a maintainable design in JavaScript:

Avoid deep inheritance hierarchies

Excess use of inheritance tends to be brittle, hard to reuse, and slow. In JavaScript, inheritance is prototypal: an object has a secret prototype link that points to its parent object. When a property isn’t found on an instance directly, the object’s prototype is searched, then the prototype’s prototype, and onwards up the inheritance chain. This can be extremely slow on objects with a long lineage. JavaScript provides more effective strategies for reuse, such as containment and dynamic mix-ins. You’re better off with a very shallow inheritance hierarchy, using these techniques to share code across objects instead.

Break code into small, independent parts

JavaScript is a functional language, and functions serve as the fundamental building block for all programs. Break complicated algorithms up into smaller functions that take only the data necessary to perform a given operation. Small functions are more easily tested, modified, and reused. What about the cost per function call that Antranig mentions? If you need to optimize for speed later, you can easily merge several functions back into one. Analyze frequent code paths and identify specific bottlenecks first, then selectively fold smaller functions into a larger one.

Don’t deeply nest functions

In JavaScript, functions can be nested inside other functions. This is a very powerful technique, since inner functions have access to the variables and scope of the outer function even after the outer function has finished executing. This is called closure, and it’s one of the best ways to encapsulate state and behaviour in JavaScript. The trick is to avoid deep nesting; don’t stick a function inside a function inside a function. Similar to the inheritance chain, each JavaScript function invocation brings with it a scope chain, providing access to variables defined in outer functions. If a reference to a particular variable isn’t found inside a function’s scope, the JavaScript runtime will climb the scope chain, searching each outer scope in turn to find the property. All this searching can get pretty slow. Limit yourself to a couple of layers of closure at the most. Here’s a slight contrived code example and an illustration of the scope chain in action:

Illustrated Code Example: Using a closure for event handling

  function showMessage(messageToDisplay) {
  	var todaysPie = "Banana creme pie";

	return function(event) {
		alert(messageToDisplay + " " + todaysPie);
		showPictureOfPie(event.target, todaysPie);
	}
  }

  var messageClickHandler = showMessage("Welcome to my pie shop. Today's pie is:")
  $(element).click(clickHandler);

Identify nameable units of behaviour and encapsulate them

On the client side, it’s really easy to let your application logic get all muddled up with your presentation code. Large, hard to maintain functions are often structured like this: 1) Perform some calculations; 2) fetch some data; 3) update the DOM. These are separate tasks, and your code will be significantly easier to read and reuse if you split these behaviours into independent units. The trick is to look at your code while you’re writing it and recognize “things” that are easily identified and named. Using a closure or an object, you can factor out pieces of the overall behaviour and data into a separate entity. Here’s an example of the various units of behaviour we identified within the Reorderer:

• The Reorderer is responsible for binding events to DOM elements and providing end-user APIs
• A Layout provides an awareness of visual layout and spatial characteristics: grids, lists, portlets within columns, and so on
• The Mover is responsible for moving elements around in the DOM, deferring to the Layout for spatial information
• A Permissions object defines the rules for valid drop targets and operations on a specific element
• Keyboard selections and accessibility is provided by the jQuery keyboard-a11y plugin
• MouseDragAndDrop encapsulates our use of jQuery UI for low-level drag and drop functionality

Use closures to encapsulate data and behaviour

Antranig covered this one really well in his post. Under many circumstances, you’ll pay a significantly lower cost to access data wrapped in a closure than within an object, and functional idioms tend to be more flexible and easier to maintain than object-oriented ones. This is particularly true in JavaScript, where there is no built-in support for namespaces or privacy. Just be careful not to nest your closures too deeply.

Never, ever cut and paste code

It almost feels silly to repeat this one here. One of the foundational principles of programming, regardless of the language you’re using, is DRY: Don’t Repeat Yourself. The bottom line is this: cut and pasted code will come back to bite you. Don’t do it. Antranig’s post hints at the idea that it’s okay to cut and past code in JavaScript. For all but the most extreme cases, I don’t buy it. If you find yourself tempted to cut and paste something, take the time to factor it into a function. It will be easier to update later, and much easier to test. Duplicate code hurts!

Recently, I expressed to Colin the sense of “disorientation” every Java (or other “traditional”) programmer must feel at diving into the client side and trying to write idiomatic, performant code. Every reasonable coder quickly develops a “gut” feeling for the costs (both in design, and resources) for the various primitives they have at their disposal. For timings, which is mainly what I am going to talk about in this posting, this might not take the form of actual hard numbers, but certainly a kind of “order of magnitude” relative estimate.

What I mean here is to know, generally within a factor of 2 or 3, how much one operation is likely to cost relative to another one, and to have a similar kind of “fuzz factor” with respect to actual performance on a particular kind of machine you are familiar with. Back in the days when I was starting on RSF, pinned to my desk was the original “How Long, Oh Lord?” which I reproduce here:

For reference, these timings were on a Windows JDK 1.4 running on a relatively weak machine in 2005, perhaps something like a c.2Ghz P4.

Anyway, to skip to the present, starting work on the RSF/Fluid client-side renderer, I felt similarly at sea on how to plan for a design that runs efficiently. Javascript is starting to emerge from its “ghetto folkloric” period and to become the target of determined engineering efforts, and there are a number of resources round the web focused on Javascript performance issues, such as John Resig’s excellent blog entry collection http://ejohn.org/blog/tags/performance/, and a splendid presentation by Julien Lecomte of YUI http://yuiblog.com/blog/2007/12/20/video-lecomte/.

Some of the “learnings” I report here in “How Long, Oh Lord 2008″ are part of Julien’s presentation, but others I have not seen before. The workhorse of these estimates is a microbenchmark collection I have put together at https://source.caret.cam.ac.uk/rsf/projects/RSFSamples/trunk/AjaxJSON/src/webapp/content/templates/perftest.html (this is just accidentally for now embedded in the middle of a wider Java/RSF/JSON/Maven development structure).

The benchmarks are run in (currently 5) “batches” of say 10,000 iterations, the most extreme measurements are discarded, and the average of the remaining three batches are reported as a headline figure. This is an attempt to defeat random variation caused by system business, GC collections and the like.

Here is a sample row and table output from the system:

Function Calls Considered Harmful

I think the most central result I want to report here is the truly fantastic expense of function calls in Javascript, which to my mind generally cuts at the heart of the possibility of orderly development.

The bad news is that a simple function call costs around 3 microseconds on a typical machine in Firefox 2.x. Opera builds are generally much better, around the 1µs mark. The “better” news going forwards is that current builds of Firefox 3 betas are vastly improved, down to 350ns on the same hardware. All the same, the ground reality is that in the vast majority of installed system for a number of years, a Javascript function call is going to cost something in the region of 1 microsecond. Let’s put that figure into perspective with a tour of our glorious history - the heroic IBM System/360 Model 30 (Announced April 7, 1964) had a machine cycle time of 1 microsecond. On the personal computer front, the Commodore PET (January 1977) was the same. In this respect (and some others) therefore Javascript sets back computer science 30 years or more. This is a little tongue-in-cheek - but not entirely. Clearly even on the most up-to-date environment, a function call in Java, say, will not execute in a single machine cycle. But the discrepancy is more like 1 order of magnitude, rather than 3 as it is in Javascript.

Why is this, and what does it mean?

Javascript function calls, as the most skilled practitioners of Javascript (Crockford, etc.) will explain, are vastly powerful, vastly underappreciated pieces of machinery. Every invocation of a Javascript function brings into existence a highly complex structure, a dynamic scope chain which will allow variables to be referenced long after a function has returned — or else, require them to be garbage collected, piece by piece. Therefore a JS “stack frame” is really very much more like an “object” in a classic “OO” language in its capabilities. More on this later. This means that a truly efficient implementation of JS function calls is essentially impossible - although the Firefox 3 team are having a thoroughly good try. What is particularly happy about this situation is that, as Crockford frequently laments, the true power of Javascript functional expressiveness is lost on 99% of its practitioners, who are typically trained if at all) in far less mind-bending idioms. And for those 1% who do truly understand what a JS function call could do, how many can make more than a handful of the function calls they write count for their full power?

What does this mean? It means that truly performant Javascript code is really obliged to drop back more than decades in its structural idiom. The function call is the primary, the first and most useful, unit of modular abstraction in programming, and these costs imply that efficient Javascript code must degrade to consist of few, extremely large function bodies, whereas a design based on fine-grained, flexibly composed units would be much more comprehensible and easy to maintain.

The most skilled Javascript programmers, framework authors, have been aware of this (implicitly or explicitly) for a long while, and it is clear that their code is following suit. As one example, TrimPath Templates a very performant macro-style templating language, is implemented with a large work-horse method (actually TrimPath uses a number of other performance tricks which will be touched upon later). As another, JQuery’s central workhorse, JQuery.find() is likewise a single monster method. In this and many other areas, “stupider is better” is often the correct answer - cut and paste, traditionally the anathema of programming, has a new lease of life in Javascript as it is frequently superior to simply repeat frequently occurring, especially small, snippets of code, rather than factor them into function calls.

Scoping and storing bugbears

The pattern of which operations in Javascript are performant and which are not is interestingly skew. To go into other results from “perftest”, we can discover a rule of thumb that one object property access is roughly equivalent to four string comparisons. For example, when testing a string value to be one of four possible values, looking it up as an Object member is roughly breaking even with writing out the sequential comparisons.

Similarly, access to objects in outer scopes carries a price. For example, again on FF2.x, simply accessing a variable not declared at the current function scope carries a penalty of 300ns. Vast, in “traditional” terms.

Facts like this militate for numerous “hoisting” tricks - essentially if a given expression or variable is accessed more than a single time in a particular function scope, it will almost always make sense to assign it to a local variable before use. This is obviously much more important for Object members than for “outer scope” variables. Julien Lecomte covers this in his presentation - a further comparison worth noticing is that function scopes are actually more more efficient containers for data than Objects. This further pushes forwards the Crockfordian view of “function scopes” as being the primary units of abstraction and storage in an “idiomatic Javascript” program, rather than traditional “Objects”. If one is going to pay 1 microsecond for a function call, better make it count - but better to pay 3 microsecond for a call and 300 ns for an access, than to pay 5 microseconds to construct an Object and then 500 ns per access (Firefox 2.x figures - FF3 and Opera are better, but the relative picture is unchanged).

An unfortunate discovery

On Firefox 2.x the already dim picture is muddied further by the discovery that simply invoking a function declared at another function scope nearly doubles the cost - even if no variables are declared or accessed at the outer scope. Since this “trick” is the now standard modern recommdation for implementing namespacing (Crockford on Privacy) this automatically penalises those who seek to organise their code better. Again blissfully this penalty is gone from Firefox 3.

Finding performant primitives

Some Javascript operations essentially retain near-native performance. Boolean tests, numeric operations, local control structures such as loops and branches, all proceed pretty quickly. This would seem to at least allow the possibility of some high-performing algorithms (assuming they were structured as giant function blocks) were it not for another really unfortunate Javascript feature - the lack of any bulk “primitive” storage. Java’s primitive array support was often widely derided as a really irritating and unorthogonal language feature, but actually is the only thing that saves it as a data processing language. In Javascript one has only Strings - and even a simple charAt() call causes a complete object instantiation (as a side tip - use of the less-well-known String.charCodeAt() followed by a (much less readable) numeric test is around 20% faster than a charAt() plus String test). There is simply nowhere to put extensive data since every Array is an Array of Object.

This leaves one to fall back on Strings and String tricks. An interesting discovery is that String.indexOf() is a startlingly rapid operation - after what appears to be a standard “native call overhead” of something like 1.5 microseconds (abouthalf a “standard function call”) (again FF 2.x), the actual search occurs at fully native speeds. This implies you can find a given String amongst around 10K of data practically as fast as you can find it amongst 100 bytes. TrimPath makes very fruitful use of indexOf to gain a very good speed boost, helped by its simple and “easily recognisable” templating format.

Regexes can take up a lot of slack in some cases, but their overhead is very much higher than an indexOf. As Lecomte notes, a Regexp.test() is *somewhat* faster than a Regexp.exec() but actually not startlingly so, since the test() appears to require no “user-visible” memory allocation. Any Regexp invocation carries a standard overhead of about 2 function calls, so use with care on simple tasks.

Garbage in, garbage out

Investigating the costs of garbage in Javascript rounds out most of this discussion. It is very interesting to repeat the performance tests on a “huge browser image” runtime, since this immediately exposes the kinds of operations which create garbage and those which do not. I am somewhat untypical in that I am frequently running a system with around 70 top-level Opera frames (it is the only browser that will cope with this sensibly) and perhaps 20 Firefox frames. In a browser image like this, the fact that all Javascript objects are typically thrown into a global shared heap, rather than the per-frame heaps one might perhaps wish for is painfully exposed. Running perftest.js in a “mega-Opera” shows the allocating tests running perhaps 100 times slower - the data becomes very spiky, but a single micro-test can end up taking a full millisecond rather than a microsecond.
This exposes that although function calls are “like” Object (heap) allocations, they are still underlyingly dealt with by different, and more efficient data structures - even in a “mega-Opera”, a bare function call is still taking 1 microsecond, whilst a true heap allocation is running at a millisecond or more. “mega-Opera” is a highly untypical environment, but it demonstrates at extremes the superior scaling costs of function frames over true garbage.

If there is a simple take-home message from all of this, it is -

i) Javascript function calls are hugely expensive compared to traditional environments, take great care not to factor your code too deeply,

ii) Javascript function calls are sufficiently powerful to take over many of the traditional roles of “Objects” in traditional environments, so consider designs that operate closure-based storage and reserve your true Objects for data transfer,

iii) make judicious use of the primitive operations which are provided (indexOf and regexps) where possible to make your code run reasonably.

We’re really close to putting out Fluid Infusion 0.3. It’ll be ready in a few days, and is shaping up to be a pretty solid release. In addition to a preview version of the Uploader, we’ve really improved the user experience and functionality of the Reorderer. We’ve added sortable layouts, customizable drag-and-drop interactions, and support for all kinds of portal-y features like locking and efficient keyboard navigation.

One of the things we’ve needed to improve in the Reorderer is the API. In earlier versions, it was pretty hard to initialize your own Reorderer. You needed to know a lot about the inner workings of objects such as LayoutHandlers, item finders, and so on. Infusion 0.3 solves this problem by providing clean, one-line functions for creating your favourite configurations of the Reorderer:

Sorting Lists

fluid.reorderList(containerSelector, itemSelector, orderChangedCallback, options);

Pass in a selector string to locate your list and the items within it, and you’ve got yourself an accessible, sortable list. Optionally, you can also specify a callback function that will be invoked every time the list order changes, along with a whole raft of other options for configuring the Reorderer.

Sorting Grids

fluid.reorderGrid(containerSelector, itemSelector, orderChangedCallback, options);

Same thing. Just pass in a selector string to locate your grid and the items within it, along with an optional order changed callback and some configuration options. Boom, a sortable 2D grid.

Sorting Layouts

fluid.reorderLayout(containerSelector, layoutSelectors, orderChangedCallback, options);

This one is for reordering portlets, content blocks, or other chunks of layout on a page. Pass in a selector to locate the container of your layout. Pass in an object that contains two selectors: one for the columns in the layout and one for the portlets. This function will do all the heavy lifting for you, figuring out which portlets are in each column, building a Layout object, and so on.

If you want all the advanced portal features of the Layout Customizer, such as locking, JSON-based layout, and permissions, you can always drop down to the lower-level API:

fluid.initLayoutCustomizer(layout, permissions, orderChangedURL, options);

This function exposes some of the more complex objects used by the Layout Customizer, the Layout and Permissions objects. The Layout is a JSON-style object representing the column and portlet structure in a DOM-agnostic way. Permissions is a complex data structure encoding all of the valid drop target permissions for each portlet and column in the layout. Generally, you’ll want to use the simpler fluid.reorderLayout() function unless you need the advanced portal features.

Documentation

We realize that code examples and the occasional blog post aren’t nearly enough to get you started using Infusion. Anastasia has been experimenting with some great new documentation for the 0.3 release. It’s still very much a work in progress, but check out her live examples of how to use the Reorderer. She’s still figuring out the right recipe for our documentation, so please share your ideas and thoughts with her!