The Fluid community shipped Infusion 1.0 a couple of weeks ago. Before making a splash with announcements about the release, we wanted to take some time to revisit our documentation and sample code to be sure they are really useful and comprehensive. Recognizing that it’s not enough just to ship a bundle of code, we try to provide a whole package of resources for developers and designers who are using Infusion. That means comprehensive API pages, tutorials to get you started, and concepts documentation to give you insights into the philosophy and design patterns we’ve built into Infusion. With this new documentation in place, now’s the time to share the news of Infusion 1.0’s release more widely.

Fluid Infusion is an application framework for building usable and accessible user interfaces with JavaScript. Built on top of jQuery, Infusion takes a different approach to client-side development. At heart, Infusion is an open architecture designed to put you back in control of your application’s user experience. It includes a growing collection of UI components—reusable interactions that go deeper than most widgets. Created by a community of developers and interaction designers, Infusion components are built from the ground up with accessibility in mind. All of our designs can be used with assistive technologies, are fully controllable with the keyboard, and can be transformed to suit your users’ personal needs.

Knowing that you can’t bottle good design, our components can be easily adapted and reworked to suit your creativity. We give you the ability to tweak more than just a few stylesheets—you can totally change the markup of a component, change its behaviour, or toss your own workflow and logic into the mix.

What’s an Application Framework?

Most JavaScript libraries fit into two categories: foundation toolkits and application frameworks. Foundation toolkits are probably the most familiar, providing all the basic tools required to manipulate the DOM, manage events, and make Ajax calls in a reliable, cross-browser way. jQuery and Prototype are probably the best-known examples. Application frameworks, on the other hand, have emerged more recently. With more complex interactions and more application logic moving from the server to the client, there is a need for tools to help manage growing code bases. Application frameworks such as Sproutcore and Dojo + Dojox often provide features such as data binding, support for the Model-View-Controller pattern, and HTML template rendering.

Infusion puts a unique spin on things: it takes a declarative and event-driven approach, promoting less code and greater flexibility. Built on top of jQuery, everything about Infusion is open: its architecture, source code, and community. Our approach to Model-View-Controller is different, too. Models and views are at the heart of Infusion, while controller code is conspicuously absent. Unlike most MVC web frameworks, we think it’s the job of the framework to take care of the glue, leaving you to focus on the stuff that makes your application tick.

Infusion’s client-side template renderer is completely unobtrusive, allowing you to dynamically render your user interface without having to mix up bits of code with your HTML. No code in your templates means more reuse, less chance for mistakes, and an easier job when it comes time to do a redesign. We also include a lightweight CSS framework that lets you mix and match styles to create beautiful and themeable user interfaces, providing pre-baked layouts, widgets and text styles to get you started. We even give you plugins and framework APIs to help ease the burden of creating user interfaces with JavaScript that will be accessible to a wide range of users.

Interoperability is huge for Infusion. We play nice with other JavaScript code, avoiding the conflicts that are common when combining different JavaScript libraries. This makes Infusion particularly well suited to building mashups, portals, and content management systems.

Fluid and jQuery

We love jQuery. Infusion wouldn’t have been possible without the simplicity and clarity of jQuery’s approach. We also didn’t want to reinvent the wheel, so jQuery is everywhere in Infusion. Recognizing that some applications need more, we created Infusion to complement jQuery and other JavaScript libraries. It’s not an either/or situation—Infusion is very modular, so you can pick and choose the features you need based on the complexity of your project. Go ahead and combine ordinary jQuery code with Infusion, or use the whole framework whenever you need robust data binding, unobtrusive markup generation, and MVC-style designs.

Fluid’s culture of collaboration and interoperability run deep. To that end, we are active contributors to jQuery UI, sharing our accessibility experience and code to help make jQuery even better. Every time we come up with a new accessibility improvement, we share it back with jQuery so that the whole community can benefit.

What’s Next?

This is just the beginning. With Infusion 1.0 out the door, we’ve got lots of new features and improvements on deck. Indeed, the past few weeks have been packed with tweaks, bug fixes, and features destined for the Infusion 1.1 release. We’re planning to ship this next release in early June. Beyond that, we’ll continue to grow our components and add new features to the framework.

We’re also active users of Infusion, building applications and helping other open source communities with their user interfaces. To this end, the Fluid community is embarking on a new project called Fluid Engage, in collaboration with museums and art galleries. We’ll be working alongside curators, education departments, and exhibit designers to help create mobile-friendly open Web solutions that bring a new level of interactivity and visitor engagement to the museum. Infusion will be the backbone of this effort, and Engage will undoubtedly drive new components and framework features back into Infusion. Keep an eye out for things like new mobile themes for the Fluid Skinning System, designed to provide a natural user experience and to repurpose content for mobile devices like the iPhone. We’ll also be shipping a new visualization library for Infusion, providing Canvas-based solutions for drawing rich, vector-based images and animations. If any of these features interests you, don’t hesitate to drop us a line if you’d like to lend a hand.

Give it a Spin and Let Us Know

We’re really excited to share Infusion with everyone. With the 1.0 release, Infusion offers a stable and reliable solution for building rich user experiences, supported by a dedicated and friendly community. We’ve got a healthy roadmap moving forward, and a growing community of users. With accessibility and usability baked in from the start, we’ve got you covered. Download the release, check out the documentation, dive in, and let us know what you think!

The code in question was kept in the file jquery-keyboard-a11y.js, and has historically always had a “special status”. This holds the parts of our framework which we have always considered closest to being contributed to the wider jQuery community (who we are always eager to develop productive relationships with), as a plugin. This code is treated specially, in that special care is made to keep it as self-contained as possible, and in particular to keep it free of dependence on our standard framework base file Fluid.js, to ease its adoption outside our community.

Unfortunately, as of release 1.5, the jQuery UI project included a file ui.selectabe.js, which registers a plugin named “selectable” with the same name as one in our set. This is interesting, since it highlights what has been considered a weakness in the jQuery plugin model - that it does not support any infrastructure for namespacing, to ensure that loosely cooperating teams can continue to work in parallel without fear of treading on each other’s names. This partly reflects its origins - the jQuery “chained filter” idiom is admirably matched for expressing queries and primitive operations on the DOM, and has produced a revolutionary condensation in the amount of code necessary to perform these “bread and butter” tasks. The plugin model is less well suited to an idiom of construction - that is, when one is using the DOM as a “springboard” for the construction of some more complex “component” or tree of objects that is going to spend some durable lifetime managing it. This can be seen from the oddity of some code which constructs say a dialog, or a sortable - where *is* the dialog in the following call?

 $("#dialog-root").dialog({bgiframe: true});

Any “object” actually referring to the dialog is implicit - the jQuery standard, is that operations on a jQuery object return another jQuery object (barring the exceptions that they are queries for primitive values such as attributes or counts) - therefore, to interact with this “dialog” we must continue to use same-named plugin calls targetted at the same DOM node (another interesting oddity is that the constructed object is not tied to the lifetime of the invoking jQuery object, but to the durability of the DOM nodes addressed by it - this is by necessity, since jQuery objects have no durability).

As a quick aside to those who have not been following these chain of blogs avidly, the Fluid framework design recommendation, since before the 0.5 release, was for “that-ism“, a code structuring device originally recommended by Douglas Crockford. I made an earlier blog posting explaining our decision, and there is also a wiki page on How to define a Unit. In brief, the benefits of that-ism include greater reliability and predictability of code - in the months since our decision we have been very happy with it, with many contributors commenting that it has made our codebase easier to understand by reading. That-ism also interacts nicely with Javascript’s natural “detached function” idiom - for example it is very easy to write a line such as

        that.viewEl.click(that.edit);

without having to worry about awkward “this” re-dispatching through event cycles etc - the edit method has natural access to its own, user-defined, and limited, context, without the need for “this juggling”. The durability of the returned that from a constructor is also matched to the created object - since it is the created object.

In any case - the jQuery model is marvellous for encoding primitive DOM operations drawn from a fixed set, slightly odd for construction, but poor for namespacing - having lost our namespace for selectable our only option is to move out of the path of the elephant. However, there is no procedure how we can guarantee that further of our names will not collide in the future, so now is an excellent opportunity to think of a scheme to make sure that our users will never again suffer this kind of API incompatibility.

Also, over the lifetime of Fluid, jQuery have firmed up their recommendations and published the following page of Guidelines for plugins - which we could do better at meeting. In particular, they recommend that each thing designated a “plugin” use exactly one name in the plugin namespace, in an attempt to head off the namespacing issues I’ve mentioned.

So, this problem is an excellent opportunity. Can we, at the same time, improve our conformance with jQuery code idioms, solve our namespacing issues permanently, and at the same time improve jquery-keyboard-a11y.js in its agreement with the that-ism enjoyed by the rest of the Fluid codebase?

It sounds impossible, but actually we can. After a lengthy brainstorming session, during which we proved that a “natural” namespacing idiom (that is, one which allows function names to be written out as Javascript symbols, rather than as string) is impossible to apply together with a “this-ist” dispatching model, we came up with the concept of the “that-ist bridge”. This is a small 20-line “machine” which now currently sits at the top of jquery-keyboard-a11y.js, and does the work of “exposing”, not only the contents of that file, but in fact the entire Fluid framework (where it has potentially conformant arguments) as a single, gigantic jQuery UI plugin named fluid, with exactly the recommended jQuery UI argument semantics.

Here is an example of our old syntax, showing invocation as a jQuery UI plugin named selectable:

$(".my-nodes").selectable({direction: $.a11y.orientation.HORIZONTAL});

“On the ground”, this has now been rewritten to a that-ist, Fluid-standard construction, used as follows:

fluid.selectable(".my-nodes", {direction: $.a11y.orientation.HORIZONTAL});

This is now housed in the global fluid namespace owned by us, permanently heading off the potential for collisions in the future. However, owing to the miracle of the “that-ist bridge” machine, this function is now also accessible, for free, using the jQuery UI plugin syntax as follows:

$(".my-nodes").fluid("selectable", {direction: $.a11y.orientation.HORIZONTAL});

Essentially any function, which accepts a “jQueryable” as its first argument (that is, anything which can legally construct a jQuery object when used as an argument to $, that is, selector strings, DOM nodes, or jQuery objects themselves) will be automatically adapted by the “that-ist Bridge” to be invocable using the single, global, Fluid jQuery UI plugin. The arguments are permuted - the first argument to the function becomes the jQuery object itself which the plugin is invoked on, and the symbol for the name of the function becomes a string, the first argument passed to the plugin. The remaining arguments to the function may be passed in an array as the 2nd plugin argument, or if there is a single argument it can be sent directly.

The treatment of return values is also somewhat interesting. The jQuery standard is that jQuery operations return further jQueries - except where they don’t. It seems that the general exceptions are those which return primitives - so this rule has been formalised in the bridge code. The return value of the underlying Fluid function is examined for its type. If it is a primitive, or “falsy” value, it is return directly. If it is some other kind of value, it is assumed that it is some kind of “Fluidic that”, and the value is hidden - instead, the original jQuery this is returned, to allow the standard jQuery selector chaining idiom to work. However, as a courtesy, the specific jQuery object returned is decorated with an extra method that() by which the originally returned value can be retrieved.

This has been a big win all round - since at the same time we have been able to improve conformance both with our own coding standards, as well as jQuery’s. And further, we have now been able to extend to the contents of this file the protection of our version management strategy, first brought in in Fluid 0.5 - by means of a namespacing trick, multiple versions of the Fluid framework may be included into the same document, whilst at the same time not only not colliding in namespace, but even remaining simultaneously addressible from the same scope by specially aware code. Truly necessity is the mother of invention. Or else, a loaf of bread.

  var myElement=$("#my-element-id");

as well as the apparently equivalent expression

  var myElement=$("[id=my-element-id]");

However, these expressions are different in many ways. One advantage of the latter is that it is more resistant to escaping issues - for example, the colon character : is actually a valid part of an XML id, but is not correctly handled in many escaping situations. JQuery will not accept it as part of a # expression, but is fine in an id specification. This was the basis of our adoption of the latter for the fluid.jById utility.

However, these types of selectors are treated very differently within the nuts-and-bolds of the jQuery selector engine. # selectors have a very special status, which enables numerous shortcuts including the judicious use of document.getElementById where possible. id selectors are just standard attribute value filters, and will perform a full scan of the DOM.

These tests were done on Firefox 2 in a rather busy document of about 40K of HTML.

Note that the cost of simply wrapping a DOM element in a jQuery object is small, but still noticeable.

Cost of guarding

This immediately suggests a policy of falling back on document.getElementById for quick lookups - however, jQuery, being a mature framework, is doing some work for us here in insulating us from an awkward IE bug where document.getElementById is sometimes returning an element which has a match on *name* rather than a match on *id*. This is a rather awkward tradeoff - putting in a “guard branch” to check that the required id is actually present as an attribute slows down access for everyone to 22µs, with the option to default back to the jQuery scan. However, do we really want a library with the contract “give me the element with this id, unless there is something screwy about your document, in which case give me the element 1000 times slower”. Probably the user will not thank you in this case, and would prefer to be given the chance to fix their markup.

Quick access to DOM nodes

This brings us back to why any of these issues are particularly interesting in the first place. I am currently working on reforming the Fluid Reorderer component, not only from a structural but also a performance point of view. The behaviour of the jQuery droppable library is not quite what we would want, both from a performance as well as a behavioural point of view (might be the subject of a separate blog posting), but in fixing it we are faced with some crucial design decisions about how to arrange data structures such that they can be used for very quick access - ordinarily this is not so important, but a DnD library is going to be queried on every mouse move event, and so even timings in microseconds can begin to mount up.

There are two main options -

i) store a “reverse lookup” cache of jQuery.data id numbers onto DOM nodes, and work with data structures storing jQuery.data ids, or
ii) synthesise short unique ids onto DOM nodes that do not have them, and work with data structures storing DOM ids.

Route i) runs the risk of keeping these caches up to date, and in particular avoiding leaks due to “expired” DOM nodes continuing to hang around in the cache - but it would probably be faster at runtime. Route ii) will be a little slower (incurs the costs we have been talking about so far, in invoking a document.getElementById equivalent on going to the DOM) but is definitely leak-safe, and might also help others navigate around the DOM.

What to do

These are hard kinds of calls to make, and involve a consideration and weighting of different kinds of issues. Currently a reasonable tradeoff for fluid.jById seems to be to go towards document.getElementById, but to put in a guard branch with a hard failure if a “name” attribute is picked up by mistake. For “live” storage during drag-and-drop, the jQuery data id cache option appears preferable, since we will already be storing various other data on drop targets in addition to what can be discovered in the DOM.

jQuery.data is really pretty fast, I will leave you with a few more timings -

Basic extending

jQuery.extend is more commonly seen as part of plugin code - the jQuery plugin model simply works by “extension” of the jQuery object itself to include new methods. For example, here is some code from the JQuery UI dialog plugin:

$.extend($.ui.dialog.overlay, {
	instances: [],
	events: $.map('focus,mousedown,mouseup,keydown,keypress,click'.split(','),
		function(e) { return e + '.dialog-overlay'; }).join(' '),
...

The extend call is taking the existing jQuery.ui.dialog.overlay object (initially set up to be a {}) and “mixing in” a set of properties and functions from an “immediate hash”.

Multiple arguments

So far, this would account for the powers of a 3-line function, and one which we have all probably independently written for ourselves (certainly in the historical forms of fluid.mixin and RSF.assign). However, where jQuery.extend really shines is in its extensions. Firstly, it will accept any number of arguments - which are just progressively mixed in on top of its first argument:

$.extend(target, source1, source2, .... sourcen);

will “blat” all the members from the source arguments serially to arrive on top of target.

Deep copy

However, the real money of jQuery.extend is in its “deepCopy” functionality. Supplying the boolean value of true as the first argument will not just copy properties laid directly on the “source” arguments, but recursively copy any properties it finds. Here is some code from the new Fluid core initialiseThat function:

      that.options = jQuery.extend(true, {}, fluid.defaults(componentName), userOptions);

This has replaced countless quantities of “options” manipulating code from around the different components, some of it quite fiddly and manual, with a single, clear, static policy in one line. The core JQuery documentation page for jQuery.extend explains this option, but in a way which might easily be missed at a quick reading. As many of the example show, it is typically used for merging “options” structures, but it is also invaluable as a plain “deep copy” operation, for example forming the basis of Fluid’s upcoming pure model idiom. Tricks like this help keep down our core library footprint.

Note on Javascript types

In such a staggeringly weakly-typed language as Javascript, this kind of useful “overloading” is generally impossible. However Javascript is at least blessed with the ability to always distinguish a handful of primitive types (string, boolean, number) from everything else, and so enables useful “Swiss army knife” methods like jQuery.extend - luckily it detectably has no meaning to try to be extending a boolean object. One way to perform this type detection is by use of the typeof operator, JQuery itself chooses to use the following test in this situation:

  if ( target.constructor == Boolean ) {

Reading through the JQuery source base is highly recommended for people with a few moments to spare, who like to know what people who know what they are doing look like

Javascript has inherited an “infusion” of “object-oriented” features grafted at the last moment on its fairly minimal and clear (LISP-inspired) baseline. The resulting language simply has too many ways of doing the same thing, and numerous features that are downright dangerous and obscure.

Continuing our chats on the performance theme, Colin and I set out to try to find a simple set of recommendations that would lead to code that is not only safe, but readable, maintainable and performant. Particularly, the aim is to cater to requirements of more structured programming, by larger, looser teams sharing components over longer timescales, rather than the “one-man”, ephemeral programs for which Javascript is more obviously suited.

In this posting I will cover a full range of “componentisation” techniques available in Javascript — from simple namespacing, structural use of closures — “scope-ism”, blending into the new “that-ism” that Fluid will promote, and contrasted against the “traditional” object-oriented language features (”this-ism”) based on this and prototype that we will deprecate, as well as more old-fashioned but performant and clear styles (”arg-ism”) carried forward from “pre-object-oriented” languages.

Open and closed

Fluid’s goal is to build platform of components which are not only flexible, but reliable and easy to extend. An important slogan in this area is “open for extension, closed for modification” which is traditionally seen in the area of object-orientation.

It is arguable whether Javascript supports object-orientation, or whether this is even a good idea, but the open/closed principle is nonetheless a good one, and clearly expresses the kind of tradeoffs we really want in an increasingly large-scale design – it should be easy to decorate the framework and components with new behaviours, but at the same time, it should be impossible to pervert or “unexpectedly change the meaning” of existing code either accidentally or deliberately. It is this latter danger, as we will see later, where Javascript’s traditional “OO” features fall down badly.

OO or not OO

Talking about “object-orientation” is traditionally dangerous, since each family of languages (C++, Java, Smalltalk, and even Javascript) brings its own particular flavours and assumptions which can make it hard to hold a regular dialogue. When we are talking about strategies for componentisation, we will want to talk more about what goals OO might achieve in an architecture rather than the exact mechanisms it might use. The three corners of OO are traditionally conceived as encapsulation, polymorphism, and inheritance – in a personal view, these have been listed in order of decreasing importance.

this-ism

Firstly we will look at the features built into the Javascript language traditionally intended to support “object orientation” and see what is wrong with them. Syntactically, these look very similar to Java’s features, but very misleadingly work nothing like them.

Here is a basic Cat:

function Cat(name) {
  this.name = name;
  }

Cat.prototype.meow = function () {
  alert(this.name + " says Meow!");
  };

var myCat = new Cat("Richard");

myCat.meow();

This is clearly “inspired” by the ability in Java to declare class Cat, but the more one looks into it, the more kinds of slippage one sees. This isn’t because the Javascript designers didn’t “understand” OO, but because they had initially set out to design a completely different language. In much the same way that Java is completely Object-focused, to the exclusion of functions as first-class citizens, Javascript takes exactly the opposite route, and is completely function-focused, barely recognising Objects at all (except as aggregates).

function Cat is called a “constructor”, or sometimes, a “constructor function”, but for a start it can be seen that very little other than convention distinguishes it from a standalone function. The use of the keyword this is mysterious – what does it refer to? This is actually a piece of magic performed by the new keyword at the time it operates the constructor function call – a new Object is created and bound to the this pointer. “new” actually does many other magic things at the same time – it also assigns to a hidden field called the “prototype” of the Object under construction, the value of another magic object called “Cat.prototype” – every Javascript function comes into existence with a prototype field, “just on the off chance” it ends up being used as a constructor (this is a little similar to the way every Java Object comes into existence with a monitor, just in case it is used as a lock :P).

This appears all well and dandy, but “this-ism” actually has so much looseness built into it as to be fatally flawed. The central problem is that every function in Javascript is “free“and hence there is no place to store the context of which object it is being invoked on. Therefore the this entry is “faked up” every time a function call is invoked, and there are no bounds that could be placed on what kind of object could be a this, or even whether it is supplied at all!

What happens if we forget to call new Cat() but simply say var MyCat = Cat()? Something terrible. For a free function not invoked through a holding Object, Javascript drops this back to hold the “global object”, which in a browser is typically the window Object – therefore this function call silently corrupts a piece of global state – there will be no compile-time or run-time warnings.

this hijacking

Similarly, nothing prevents “this hijacking” – for example it is the tiniest effort in the world to follow the code above with this perversion:


var myDog = new Dog();
myDog.meow = myCat.meow;
myDog.meow();


The assumptions, whatever they were, in the “meow” function above have probably been violated – since Javascript is a “typeless” language it would be hard to write down what these assumptions might be in order to enforce them – and if myDog happened to have a field called “name” the meow would probably succeed. However, the inability to ever know, when writing (or even reading) a piece of code, which value one can expect to be bound to the “this” pointer makes it impossible to write reliable and predictable code with this idiom. A flavour of the kinds of contortions “this-ism” forces on coders can be seen when dealing with (browser) event handlers – since these always *are* pure functions, the “this” context of any previously bound “object methods” which are assigned as handlers becomes lost, and the better Javascript frameworks pride themselves on doing automatic “this adjustment” before dispatching to a handler. This sort of thing is fine for frameworks, but is the sort of thing that can easily be “accidentally” foisted onto a user as soon as they come to write new components for a framework.

Note on function naming - capitalisation for constructors

Note that a useful convention in naming Javascript functions is to reserve those beginning with capital letters only for old-fashioned “this-ist” constructors - this can help to head off this-ist disasters caused by forgetting to invoke those functions which require it with new. So we write Cat for the constructor we expect to be used with new Cat(), but later on we will write cat or make_cat for functions with a “constructive effect” that don’t require new.

Prototype hijacking

A final “perversion” following from this-ism is of “prototype hijacking”. Many of the effects (and many unrelated effects) of OO “inheritance” can be emulated in Javascript “this-ism” by means of assembling a chain of prototype instances.

SuperCat.prototype = new Cat();

function SuperCat(name) {
  Cat.call(name);
  this.superpower = "fly";
  }

var mySuperCat = new SuperCat("Edward");

mySuperCat.meow();

This is powerful, just as inheritance is in classically OO languages. However, it is “too powerful” – since we have violated the Open/Closed principle. Having happily written our own inheritance hierarchy of Cats, perfectly legal Javascript may subsequently come along and write SuperCat.prototype = new Dog().

This not only has the devastating effect of causing future SuperCats to in fact become Dogs, it futhermore causes all SuperCats ever created to also retrospectively become Dogs. Thus this changes the meaning not only of the code within SuperCat methods, but the meaning of all client code that has ever used a SuperCat.

This is “cool” – in the sense that it is a really cool capability to impress your friends who use other languages – and a cool capability to use in code which just one person is working on (you), when that person has a really good memory and a clear head. It is far from cool when working on a large team, perhaps not all of whom one knows very well, and whose code one has to read and maintain over a long period.

Code that is easy to follow and understand is just another side of the same coin which Fluid calls “portal-friendliness” – code is “portal safe” if it does not have “unexpected” collateral effects on other code, for example by assigning to core language primitives, or similarly in this case changing the meaning of 3rd-party code by dicking around with its inheritance structure.

Security

If you believe that a Javascript program can in any way be “secure” (which I don’t, given current environments and practices) the above “hijacks” and “perversions” can also be seen to be dangerous from a security perspective, in that they allow unintended changes in meaning to 3rd-party code. In upcoming environments such as Google’s Caja, actually secure coding will start to be possible – and it is no coincidence that the first language features to meet the axe will be this and prototype. In
the meantime, you can get a start at this-less coding before the revolution comes, and enjoy at least what security and reliability today’s Javascript has to offer.

Arg-ism

Having considered that Javascript’s OO features are unusable in the large, we banish this whole infrastructure from our minds (including this, new, prototype and constructor) and begin to look for other ways of coding that will let us achieve some of the same goals.

The first question to ask is, “do we really need Object Orientation?”, or more accurately, how could we achieve some of its goals without any particular real assistance from the language?

The object-oriented features that C has to offer

My friend Bob and I were once C++ programmers, in the era when C++ held the role as the “acceptable public face of OO” – worthy, slightly stodgy, and with possibly worrying performance characteristics. A favorite joke we would share was a phrase once dropped by a particularly capable “old salt” programmer who once said that given the choice between C and C++, that he “preferred the object oriented features that C had to offer”.

Was this a joke? Is that a question? What *are* the object oriented features that C has to offer? Luckily, whatever they are, they are available in most languages ever written, and many people are indeed happy with them.

A dirty secret is that the “this” pointer we are familiar with from C++ and Java is really just a “hidden function call argument”. And indeed, the OO calls we started to see in 80’s C++ evolved directly from their equivalent 70’s C calls that looked like this:

typedef struct Cat {
  char* name;
  } Cat;

void meow(Cat* this) {
  printf("%s says Meow!", this->name);
  }

Cat myCat = {"Richard"};

meow(myCat);

Oh the good old days. What have we lost here from “OO”? Well, we have lost some encapsulation – we cannot hide the details of the struct members from the public, but we could never do that with “this-ism” either. So that is no loss. What we have lost is polymorphism – the meow function is statically dispatched, and could not be “overridden” to behave differently based on the contents of the this pointer it was handed. In the C days this sort of thing was done with type fields and switch statements, and generally fell down in the “Open-ness” stakes – these designs are not sufficiently in some cases “open for extension” but at least they don’t fail to be “closed for modification”. Noone could send a non-Cat to our meow function.

However in many cases polymorphism is simply not appropriate in a design – so why should we have to always pay for it? This was one of the early criticisms levelled against C++, which was reponsible for its rather “neither fish nor fowl” OO system that didn’t actually give you polymorphic dispatch unless you asked for it. But what this demonstrates is that this is a perfectly valid compositional technique for many situations – and it has the advantages that it is extremely performant, and easy to follow:

function makeCat(name) {
  var togo = {name: name};
  }

function cat_meow(cat) {
  alert(cat.name + " says Meow!");
  };

var myCat = makeCat("Richard");

cat_meow(myCat);

In this little example we see no suffering imposed through “arg-ism” and the code is much clearer and will run faster. In a, perhaps strongly “data-driven”, design where you are sure no polymorphism or extension through inheritance will be required, C-style componentisation is an approach strongly to be considered. For example, the RSF component tree idiom is “closed” by design – despite forming a hierarchy, it is a hierarchy which is designed purely to classify data, and is “closed” in intent, since user code is not intended to be able to create new “component types”. Arg-ism would be appropriate for representing and constructing components of this type.

Scope-ism

None of our design options so far have actually succeeded on delivering on arguably the most important of the 3 cornerstones of OO, encapsulation. Protecting data from unexpected and asynchronous modification is one of the most important things we can do to make a design more reliable and easier to maintain, through making it easier to reason about the kinds of effects a piece of code might have, and what it could influence.

As most famously proposed by Douglas Crockford, the only means of achieving properly private data in Javascript is to go back to its functional roots, and hide these within function closures. If you are not familiar with Javascript’s closure support, now would be a good time to browse around several of the excellent articles on Crockford’s site to familiarise yourself.

Scope-ism for modules

This is a structuring device which can be used at all levels within a Javascript program. Sometimes the depth of nesting can become somewhat dizzying, but adopting a consistent indentation and coding style can help a lot with this. At the outermost level, we can gain encapsulation for an entire “module” by enclosing wide swaths of Javascript code in a closure:

var Fluid = Fluid || {};

(function () {
  function myPrivateFunction (arg) {
    alert("Your arg was " + arg);
    }

Fluid.myPublicFunction = function() {
    return myPrivateFunction("thing");
    }

  }) ();

[Note that this, superior, style of module declaration has come to be called "Colinism" – it is a slight variation on the pattern seen in Crockford and other places in that the public namespace object can be "accreted" – it is "guarded" in the first line to ensure that it exists and is at global scope, and then in the anonymous function block it has public members assigned to it one by one]

The definition myPrivateFunction is completely inaccessible to code outside this block, as would any data or other material which was declared in this area. Finally we can achieve “encapsulation”, here at the level of an entire “module”. Only values which “escape” by being assigned to globally visible variables as a result of the block execution can be seen by other code. The global Fluid object can itself be corrupted by having its members modified, but at least the code we write inside myPublicFunction now has the guarantees of a stable environment. If the code executes at all, it will generally do so in a tamper-proof box that makes it much easier to debug and maintain.

Scope-ism in the smaller

This concept of a “module” is really one again established by convention – Javascript is a “do it yourself” language and all these structuring devices are just patterns we agree amongst ourselves. The same kind of encapsulation is also invaluable on smaller scales, creating a kind of structure which doesn’t really have a name, but is quite typical, for example, in “init blocks” which fire on page loads and traverse the DOM to contextualise and attach event handlers to what they find – for example here is the skeleton of an init block I recently wrote:

  AddParticipants.initChooseRoles = function (baseID, selfTemplate) {
    var base = $jit(baseID);
    var IDtoIndex = {};
    var indexToRow = [];
    indexTable("user-key", base, "tr.role-row", IDtoIndex, indexToRow);
    var checks = base.find(".role-row input");

    var lastChecked; // The "true ID" for the DOM element  

    checks.click(function(event) {
      var checked = $(event.target).attr("checked");
      if (checked) {
        if (lastChecked && event.shiftKey) {
          var index1 = IDtoIndex[lastChecked];
          var index2 = IDtoIndex[getIDFromDerived(event.target, keyField)];
           ....

}

We can see the outer level of this function as following the overall module pattern from the last section – the function “initChooseRoles” is attached to the global namespace object AddParticipants which is similarly guarded to “Fluid” from before. This function now begins by setting up some shared definitions, having found its targets within the DOM – then these definitions, like base, IDtoIndex, indexToRow and lastChecked become “closed over” by numerous inner function definitions (typically handlers). We can see this inner function block as a kind of “Object” of its own, with private definitions and public effects separated out.

Scope-ism in the smallest – that-ism

Our final destination in this journey through Javascript structuring devices is also due to Crockford, is the smallest-scale use of function scopes, that for direct replacement of the faulty “this-ism” where we came in. That-ism is alluded to by Crockford on the private members page linked above, but is more fully developed in his recent book Javascript: The Good Parts in Chapter 5 (all of this book is thoroughly recommended reading).

The answer to a fragile “identity” (this) for object methods is to replace them with a function-scoped alternative, which for want of a better non-reserved alternative, Crockford calls “that”. “That-ism” looks extremely similar to the function scoped examples we have seen before – with only a difference of emphasis. The difference is that we not only close over essentially *one* piece of state, holding the role formerly occupied by a this, but that we also return it from the function itself. So our “that-ified” cat example looks as follows:

function cat(name) {
  var that = {};
  that.name = name;
  that.meow = function () {
    alert(that.name + " says Meow!");

    };

  return that;
  }
var myCat = cat("Richard");

myCat.meow();

Compare the differences between “that cat” and “this cat”. The usage is very similar, only we no longer use the pernicious keyword “this”. However the definition style is quite different – not being able to leverage prototypalism, we need to physically bind all the CAT methods we require onto each cat instance as we dispense it. In many ways this is “more” “object-oriented” than what Javascript supplies out of the box, since we have regained the 3rd leg of encapsulation – we could write any number of “private variables” and definitions inside the cat() body which would be firmly hidden.

On the face of it this looks like we have had to sacrifice inheritence – but to a large extent we can get this back too. Here is our happy SuperCat:

function superCat(name) {
  var that = cat(name);
  that.superpower = "fly";
  return that;
  }

var mySuperCat = superCat("Edward");

mySuperCat.meow();

Simply by “instance chaining” we can cascade that-ism from “constructor” to “superconstructor” and obtain most of the (useful) effects of inheritance. Our design is “open for extension” again.

Drawbacks of that-ism

The only really significant drawback of that-ism is efficiency. We are literally fabricating new function handles for each of its methods every time we manufacture an object, and this has to be contrasted with the efficiency of this-ism and prototypalism whereby the selfsame function handles are shared amongst all instance of a “type”. In theory this overhead could be quite small – the “closed-over” data in each method is just the that reference, and with the space for the function handle itself, say in a Java-type VM this would amount to 8n or 16n bytes for an object with n methods. Whether we consider this significant or not would tend to depend on the context – where a “component” refers to a user-visible widget that is rendered in a DOM, this expense would be dwarfed by any browser-side or OS-side resources required to represent it. In a more resource-intensive role (perhaps for rendering or data processing) it would probably be unacceptable, and one would be more prone to fall back upon the most efficient scheme of all, the glorious arg-ism of the 70s.

Conclusion

What to use and when?

The general recommendation of this article is “use that, not this”. This-ism has been argued to be almost universally destructive and not even deliver on the full goals that were staked out for “object oriented” designs. For some applications, expecially high-performance and/or data-driven applications, users have been recommended to fall back on “the object-oriented features which C has to offer”, being fully argument-driven binding (arg-ism), and where any polymorphism is required to achieve this through duck typing or other data-driven idioms. And overall, to play to Javascript’s (original) strengths – use function block scopes for storing data and creating “units with identity” (objects) rather than Javascript Objects themselves – which last was also the final advice from my performance-oriented posting.

As I speak, Fluid is fully invested in a wholesale conversion from this-ism to that-ism, for which there have been presentation materials at the latest “Fearless Javascript” workshops, and also a growing body of that-ism documentation on the Fluid wiki. Take that!

The workshop explores what I’ve been referring to as “modern” JavaScript programming, for lack of a better term. In contrast to the previous generation of JavaScript programs, where small bits of interactivity were sprinkled on top of a largely static document, modern JavaScript involves the creation of richly interactive applications and components. These days, your code probably shares the page and Javascript’s notorious global namespace with other libraries and widgets. As a result, you need to do things a bit differently to ensure your code is self-contained and won’t conflict with other libraries. Fearless JavaScript gives you the techniques needed to thrive in the world of portals, plugins, and mash-ups.

We’ll be presenting a new edition of Fearless JavaScript at the Sakai Summer 2008 conference in Paris, France. The material has been significantly refined, and we’ve added another co-presenter: Nicolaas Matthijs from CARET at the University of Cambridge. Nico will be talking about how to create gadgets for Sakai using JSON-based data feeds. We’re working on some tutorial code that walks you through using Fluid components within a Sakai gadget. Cool stuff.

Nico attended the original Fearless JavaScript workshop in St. Paul, and had lots of interesting questions and suggestions for us. Afterwards, Nico and I spent an enjoyable couple of hours working together on some of his gadgets, tweaking their accessibility and talking about his designs. He’s doing some great work on MyCamTools, and I’m looking forward to sharing the stage with him and Eli for Fearless JavaScript 2.0.

If you’re in Paris for the Sakai conference, we’d love to see you at the workshop. If you can’t make it, we’re hoping to create a podcast of the event. Here are the details:

Fearless JavaScript
Presented by Colin Clark, Eli Cochran, and Nicolaas Matthijs
Monday, June 30
1:30 - 4:30 pm in Amphi 55A

Topics will include:

* Why JavaScript?
* Fundamentals of JavaScript
* Using jQuery
* JavaScript in Sakai
* Portal and mash-up friendliness
* Accessibility

Everyone is welcome to attend the workshop. It’s assumed that you have some understanding of at least one programming language, and that you’ll bring a laptop to follow along with the hands-on exercises.

See you there!

“Premature optimization is the root of all evil.”

Antranig’s performance metrics are intended to help establish a gut feel about the cost of a particular operation. They’re low-level measurements, and as such they can’t take into account a fundamental variable: context. These performance metrics have very different consequences depending on the size, scale, and nature of the application. His examples are apt: intensive, highly recursive, up-front operations like template rendering show the full cost of JavaScript’s weak performance. Indeed, Antranig has used his research to good effect on our nascent client-side template renderer. In a relatively short amount of time, he was able to improve the performance of his low-level parsing algorithms by several orders of magnitude using regular expressions and fast string comparisons. Impressive stuff.

On the other hand, context matters a lot when it comes to user-facing behaviour, where timings may vary significantly based on the user’s choices. Many UI components, such as the Uploader, provide a fairly modal experience for the user. Such components undoubtedly have less stringent performance requirements, and our acceptance criteria should be determined by testing in real-world situations.

The risk with out of context performance measurements is the tendency to optimize prematurely and without accurate data. There’s only so much time in the day. Don’t lose the opportunity to add useful features to your application by starting in on time-intensive performance optimizations too early. Without good data, your hard work may only have a tiny impact on the overall perception of speediness. Worse yet, premature optimization can lead towards architectures that are poorly factored, redundant, and hard to maintain.

So how do we ensure good performance without compromising architectural quality?

1. Write code with good design principles in mind
2. Continually measure performance in real world situations
3. Refine code incrementally to improve performance

Let’s take jQuery as an example. If you look at the first release of jQuery, you’ll find that it was mostly composed of small functions that are clear and easy to read. jQuery’s author, John Resig, has always done a very good job of carefully profiling his code with performance tests, and over time the code base has evolved in certain areas to be faster and, as a concequence, occasionally less readable. He took the time to build something real first, then he optimized strategically. This is exactly the example of how we should all handle performance optimizations: build it cleanly, then identify areas where performance optimizations are necessary. Optimization without real-world numbers is, well, evil.

Building on Antranig’s advice, here are some strategies to help ensure both good performance and a maintainable design in JavaScript:

Avoid deep inheritance hierarchies

Excess use of inheritance tends to be brittle, hard to reuse, and slow. In JavaScript, inheritance is prototypal: an object has a secret prototype link that points to its parent object. When a property isn’t found on an instance directly, the object’s prototype is searched, then the prototype’s prototype, and onwards up the inheritance chain. This can be extremely slow on objects with a long lineage. JavaScript provides more effective strategies for reuse, such as containment and dynamic mix-ins. You’re better off with a very shallow inheritance hierarchy, using these techniques to share code across objects instead.

Break code into small, independent parts

JavaScript is a functional language, and functions serve as the fundamental building block for all programs. Break complicated algorithms up into smaller functions that take only the data necessary to perform a given operation. Small functions are more easily tested, modified, and reused. What about the cost per function call that Antranig mentions? If you need to optimize for speed later, you can easily merge several functions back into one. Analyze frequent code paths and identify specific bottlenecks first, then selectively fold smaller functions into a larger one.

Don’t deeply nest functions

In JavaScript, functions can be nested inside other functions. This is a very powerful technique, since inner functions have access to the variables and scope of the outer function even after the outer function has finished executing. This is called closure, and it’s one of the best ways to encapsulate state and behaviour in JavaScript. The trick is to avoid deep nesting; don’t stick a function inside a function inside a function. Similar to the inheritance chain, each JavaScript function invocation brings with it a scope chain, providing access to variables defined in outer functions. If a reference to a particular variable isn’t found inside a function’s scope, the JavaScript runtime will climb the scope chain, searching each outer scope in turn to find the property. All this searching can get pretty slow. Limit yourself to a couple of layers of closure at the most. Here’s a slight contrived code example and an illustration of the scope chain in action:

Illustrated Code Example: Using a closure for event handling

  function showMessage(messageToDisplay) {
  	var todaysPie = "Banana creme pie";

	return function(event) {
		alert(messageToDisplay + " " + todaysPie);
		showPictureOfPie(event.target, todaysPie);
	}
  }

  var messageClickHandler = showMessage("Welcome to my pie shop. Today's pie is:")
  $(element).click(clickHandler);

Identify nameable units of behaviour and encapsulate them

On the client side, it’s really easy to let your application logic get all muddled up with your presentation code. Large, hard to maintain functions are often structured like this: 1) Perform some calculations; 2) fetch some data; 3) update the DOM. These are separate tasks, and your code will be significantly easier to read and reuse if you split these behaviours into independent units. The trick is to look at your code while you’re writing it and recognize “things” that are easily identified and named. Using a closure or an object, you can factor out pieces of the overall behaviour and data into a separate entity. Here’s an example of the various units of behaviour we identified within the Reorderer:

• The Reorderer is responsible for binding events to DOM elements and providing end-user APIs
• A Layout provides an awareness of visual layout and spatial characteristics: grids, lists, portlets within columns, and so on
• The Mover is responsible for moving elements around in the DOM, deferring to the Layout for spatial information
• A Permissions object defines the rules for valid drop targets and operations on a specific element
• Keyboard selections and accessibility is provided by the jQuery keyboard-a11y plugin
• MouseDragAndDrop encapsulates our use of jQuery UI for low-level drag and drop functionality

Use closures to encapsulate data and behaviour

Antranig covered this one really well in his post. Under many circumstances, you’ll pay a significantly lower cost to access data wrapped in a closure than within an object, and functional idioms tend to be more flexible and easier to maintain than object-oriented ones. This is particularly true in JavaScript, where there is no built-in support for namespaces or privacy. Just be careful not to nest your closures too deeply.

Never, ever cut and paste code

It almost feels silly to repeat this one here. One of the foundational principles of programming, regardless of the language you’re using, is DRY: Don’t Repeat Yourself. The bottom line is this: cut and pasted code will come back to bite you. Don’t do it. Antranig’s post hints at the idea that it’s okay to cut and past code in JavaScript. For all but the most extreme cases, I don’t buy it. If you find yourself tempted to cut and paste something, take the time to factor it into a function. It will be easier to update later, and much easier to test. Duplicate code hurts!

What I mean here is to know, generally within a factor of 2 or 3, how much one operation is likely to cost relative to another one, and to have a similar kind of “fuzz factor” with respect to actual performance on a particular kind of machine you are familiar with. Back in the days when I was starting on RSF, pinned to my desk was the original “How Long, Oh Lord?” which I reproduce here:

For reference, these timings were on a Windows JDK 1.4 running on a relatively weak machine in 2005, perhaps something like a c.2Ghz P4.

Anyway, to skip to the present, starting work on the RSF/Fluid client-side renderer, I felt similarly at sea on how to plan for a design that runs efficiently. Javascript is starting to emerge from its “ghetto folkloric” period and to become the target of determined engineering efforts, and there are a number of resources round the web focused on Javascript performance issues, such as John Resig’s excellent blog entry collection http://ejohn.org/blog/tags/performance/, and a splendid presentation by Julien Lecomte of YUI http://yuiblog.com/blog/2007/12/20/video-lecomte/.

Some of the “learnings” I report here in “How Long, Oh Lord 2008″ are part of Julien’s presentation, but others I have not seen before. The workhorse of these estimates is a microbenchmark collection I have put together at https://source.caret.cam.ac.uk/rsf/projects/RSFSamples/trunk/AjaxJSON/src/webapp/content/templates/perftest.html (this is just accidentally for now embedded in the middle of a wider Java/RSF/JSON/Maven development structure).

The benchmarks are run in (currently 5) “batches” of say 10,000 iterations, the most extreme measurements are discarded, and the average of the remaining three batches are reported as a headline figure. This is an attempt to defeat random variation caused by system business, GC collections and the like.

Here is a sample row and table output from the system:

Function Calls Considered Harmful

I think the most central result I want to report here is the truly fantastic expense of function calls in Javascript, which to my mind generally cuts at the heart of the possibility of orderly development.

The bad news is that a simple function call costs around 3 microseconds on a typical machine in Firefox 2.x. Opera builds are generally much better, around the 1µs mark. The “better” news going forwards is that current builds of Firefox 3 betas are vastly improved, down to 350ns on the same hardware. All the same, the ground reality is that in the vast majority of installed system for a number of years, a Javascript function call is going to cost something in the region of 1 microsecond. Let’s put that figure into perspective with a tour of our glorious history - the heroic IBM System/360 Model 30 (Announced April 7, 1964) had a machine cycle time of 1 microsecond. On the personal computer front, the Commodore PET (January 1977) was the same. In this respect (and some others) therefore Javascript sets back computer science 30 years or more. This is a little tongue-in-cheek - but not entirely. Clearly even on the most up-to-date environment, a function call in Java, say, will not execute in a single machine cycle. But the discrepancy is more like 1 order of magnitude, rather than 3 as it is in Javascript.

Why is this, and what does it mean?

Javascript function calls, as the most skilled practitioners of Javascript (Crockford, etc.) will explain, are vastly powerful, vastly underappreciated pieces of machinery. Every invocation of a Javascript function brings into existence a highly complex structure, a dynamic scope chain which will allow variables to be referenced long after a function has returned — or else, require them to be garbage collected, piece by piece. Therefore a JS “stack frame” is really very much more like an “object” in a classic “OO” language in its capabilities. More on this later. This means that a truly efficient implementation of JS function calls is essentially impossible - although the Firefox 3 team are having a thoroughly good try. What is particularly happy about this situation is that, as Crockford frequently laments, the true power of Javascript functional expressiveness is lost on 99% of its practitioners, who are typically trained if at all) in far less mind-bending idioms. And for those 1% who do truly understand what a JS function call could do, how many can make more than a handful of the function calls they write count for their full power?

What does this mean? It means that truly performant Javascript code is really obliged to drop back more than decades in its structural idiom. The function call is the primary, the first and most useful, unit of modular abstraction in programming, and these costs imply that efficient Javascript code must degrade to consist of few, extremely large function bodies, whereas a design based on fine-grained, flexibly composed units would be much more comprehensible and easy to maintain.

The most skilled Javascript programmers, framework authors, have been aware of this (implicitly or explicitly) for a long while, and it is clear that their code is following suit. As one example, TrimPath Templates a very performant macro-style templating language, is implemented with a large work-horse method (actually TrimPath uses a number of other performance tricks which will be touched upon later). As another, JQuery’s central workhorse, JQuery.find() is likewise a single monster method. In this and many other areas, “stupider is better” is often the correct answer - cut and paste, traditionally the anathema of programming, has a new lease of life in Javascript as it is frequently superior to simply repeat frequently occurring, especially small, snippets of code, rather than factor them into function calls.

Scoping and storing bugbears

The pattern of which operations in Javascript are performant and which are not is interestingly skew. To go into other results from “perftest”, we can discover a rule of thumb that one object property access is roughly equivalent to four string comparisons. For example, when testing a string value to be one of four possible values, looking it up as an Object member is roughly breaking even with writing out the sequential comparisons.

Similarly, access to objects in outer scopes carries a price. For example, again on FF2.x, simply accessing a variable not declared at the current function scope carries a penalty of 300ns. Vast, in “traditional” terms.

Facts like this militate for numerous “hoisting” tricks - essentially if a given expression or variable is accessed more than a single time in a particular function scope, it will almost always make sense to assign it to a local variable before use. This is obviously much more important for Object members than for “outer scope” variables. Julien Lecomte covers this in his presentation - a further comparison worth noticing is that function scopes are actually more more efficient containers for data than Objects. This further pushes forwards the Crockfordian view of “function scopes” as being the primary units of abstraction and storage in an “idiomatic Javascript” program, rather than traditional “Objects”. If one is going to pay 1 microsecond for a function call, better make it count - but better to pay 3 microsecond for a call and 300 ns for an access, than to pay 5 microseconds to construct an Object and then 500 ns per access (Firefox 2.x figures - FF3 and Opera are better, but the relative picture is unchanged).

An unfortunate discovery

On Firefox 2.x the already dim picture is muddied further by the discovery that simply invoking a function declared at another function scope nearly doubles the cost - even if no variables are declared or accessed at the outer scope. Since this “trick” is the now standard modern recommdation for implementing namespacing (Crockford on Privacy) this automatically penalises those who seek to organise their code better. Again blissfully this penalty is gone from Firefox 3.

Finding performant primitives

Some Javascript operations essentially retain near-native performance. Boolean tests, numeric operations, local control structures such as loops and branches, all proceed pretty quickly. This would seem to at least allow the possibility of some high-performing algorithms (assuming they were structured as giant function blocks) were it not for another really unfortunate Javascript feature - the lack of any bulk “primitive” storage. Java’s primitive array support was often widely derided as a really irritating and unorthogonal language feature, but actually is the only thing that saves it as a data processing language. In Javascript one has only Strings - and even a simple charAt() call causes a complete object instantiation (as a side tip - use of the less-well-known String.charCodeAt() followed by a (much less readable) numeric test is around 20% faster than a charAt() plus String test). There is simply nowhere to put extensive data since every Array is an Array of Object.

This leaves one to fall back on Strings and String tricks. An interesting discovery is that String.indexOf() is a startlingly rapid operation - after what appears to be a standard “native call overhead” of something like 1.5 microseconds (abouthalf a “standard function call”) (again FF 2.x), the actual search occurs at fully native speeds. This implies you can find a given String amongst around 10K of data practically as fast as you can find it amongst 100 bytes. TrimPath makes very fruitful use of indexOf to gain a very good speed boost, helped by its simple and “easily recognisable” templating format.

Regexes can take up a lot of slack in some cases, but their overhead is very much higher than an indexOf. As Lecomte notes, a Regexp.test() is *somewhat* faster than a Regexp.exec() but actually not startlingly so, since the test() appears to require no “user-visible” memory allocation. Any Regexp invocation carries a standard overhead of about 2 function calls, so use with care on simple tasks.

Garbage in, garbage out

Investigating the costs of garbage in Javascript rounds out most of this discussion. It is very interesting to repeat the performance tests on a “huge browser image” runtime, since this immediately exposes the kinds of operations which create garbage and those which do not. I am somewhat untypical in that I am frequently running a system with around 70 top-level Opera frames (it is the only browser that will cope with this sensibly) and perhaps 20 Firefox frames. In a browser image like this, the fact that all Javascript objects are typically thrown into a global shared heap, rather than the per-frame heaps one might perhaps wish for is painfully exposed. Running perftest.js in a “mega-Opera” shows the allocating tests running perhaps 100 times slower - the data becomes very spiky, but a single micro-test can end up taking a full millisecond rather than a microsecond.
This exposes that although function calls are “like” Object (heap) allocations, they are still underlyingly dealt with by different, and more efficient data structures - even in a “mega-Opera”, a bare function call is still taking 1 microsecond, whilst a true heap allocation is running at a millisecond or more. “mega-Opera” is a highly untypical environment, but it demonstrates at extremes the superior scaling costs of function frames over true garbage.

If there is a simple take-home message from all of this, it is -

i) Javascript function calls are hugely expensive compared to traditional environments, take great care not to factor your code too deeply,

ii) Javascript function calls are sufficiently powerful to take over many of the traditional roles of “Objects” in traditional environments, so consider designs that operate closure-based storage and reserve your true Objects for data transfer,

iii) make judicious use of the primitive operations which are provided (indexOf and regexps) where possible to make your code run reasonably.

One of the things we’ve needed to improve in the Reorderer is the API. In earlier versions, it was pretty hard to initialize your own Reorderer. You needed to know a lot about the inner workings of objects such as LayoutHandlers, item finders, and so on. Infusion 0.3 solves this problem by providing clean, one-line functions for creating your favourite configurations of the Reorderer:

Sorting Lists

fluid.reorderList(containerSelector, itemSelector, orderChangedCallback, options);

Pass in a selector string to locate your list and the items within it, and you’ve got yourself an accessible, sortable list. Optionally, you can also specify a callback function that will be invoked every time the list order changes, along with a whole raft of other options for configuring the Reorderer.

Sorting Grids

fluid.reorderGrid(containerSelector, itemSelector, orderChangedCallback, options);

Same thing. Just pass in a selector string to locate your grid and the items within it, along with an optional order changed callback and some configuration options. Boom, a sortable 2D grid.

Sorting Layouts

fluid.reorderLayout(containerSelector, layoutSelectors, orderChangedCallback, options);

This one is for reordering portlets, content blocks, or other chunks of layout on a page. Pass in a selector to locate the container of your layout. Pass in an object that contains two selectors: one for the columns in the layout and one for the portlets. This function will do all the heavy lifting for you, figuring out which portlets are in each column, building a Layout object, and so on.

If you want all the advanced portal features of the Layout Customizer, such as locking, JSON-based layout, and permissions, you can always drop down to the lower-level API:

fluid.initLayoutCustomizer(layout, permissions, orderChangedURL, options);

This function exposes some of the more complex objects used by the Layout Customizer, the Layout and Permissions objects. The Layout is a JSON-style object representing the column and portlet structure in a DOM-agnostic way. Permissions is a complex data structure encoding all of the valid drop target permissions for each portlet and column in the layout. Generally, you’ll want to use the simpler fluid.reorderLayout() function unless you need the advanced portal features.

Documentation

We realize that code examples and the occasional blog post aren’t nearly enough to get you started using Infusion. Anastasia has been experimenting with some great new documentation for the 0.3 release. It’s still very much a work in progress, but check out her live examples of how to use the Reorderer. She’s still figuring out the right recipe for our documentation, so please share your ideas and thoughts with her!

“Jess comes with an incredibly rich and relevant background in managing large, complex, distributed projects. She is very familiar with the academic environment, the schools engaged in Fluid and the open source and community source development process. Until recently, Jess was the Innovative Projects Manager at Duke University and managed, among many innovative projects, the Duke Digital Initiative.”

I’m impressed with how diverse Jess’ background is. She has a firm grasp of technical issues and was a volunteer with the Geekcorps project to build independent network infrastructure in Ghana. She’s managed a variety of projects in a higher ed setting, and has taught project courses at Duke University. Her academic background in Political Science and Philosophy brings a unique perspective on community building and open source culture. And I know she’s been learning a ton about inclusive design, too!

A quick bit of history: Fluid has been trying to hire a dedicated Project Manager for over a year now. Hiring at a university can be pretty slow and bureaucratic, and we had several promising candidates who were unexpectedly wooed away by offers from the private sector. Throughout this process, we took the time to be sure that we hired someone who will thrive within the unique constraints of a distributed, community-driven environment. I’m thoroughly confident that Jess will excel in this role, and she is committed to helping us communicate and collaborate better as our community grows.

On a personal level, I’m relieved by the prospect of being able to focus on technical leadership for Fluid. With all the task-juggling and cat-herding I’ve done over the past year and a half, it’s been hard to put enough attention towards architecture and coding. Now that Jess is on board, I’ll be much more focused on development work. I have a backlog of technical writing and coding to catch up with, so expect to hear a lot more from me on the lists and this blog with code examples, screenshots of new components, and more.

Jess will be based in Boston, but is planning to do a fair bit of traveling over the next few months. She’s also really interested in hearing from members of the community, so don’t hesitate to drop her a line. Please join me in welcoming Jess to the Fluid community!