The workshop explores what I’ve been referring to as “modern” JavaScript programming, for lack of a better term. In contrast to the previous generation of JavaScript programs, where small bits of interactivity were sprinkled on top of a largely static document, modern JavaScript involves the creation of richly interactive applications and components. These days, your code probably shares the page and Javascript’s notorious global namespace with other libraries and widgets. As a result, you need to do things a bit differently to ensure your code is self-contained and won’t conflict with other libraries. Fearless JavaScript gives you the techniques needed to thrive in the world of portals, plugins, and mash-ups.

We’ll be presenting a new edition of Fearless JavaScript at the Sakai Summer 2008 conference in Paris, France. The material has been significantly refined, and we’ve added another co-presenter: Nicolaas Matthijs from CARET at the University of Cambridge. Nico will be talking about how to create gadgets for Sakai using JSON-based data feeds. We’re working on some tutorial code that walks you through using Fluid components within a Sakai gadget. Cool stuff.

Nico attended the original Fearless JavaScript workshop in St. Paul, and had lots of interesting questions and suggestions for us. Afterwards, Nico and I spent an enjoyable couple of hours working together on some of his gadgets, tweaking their accessibility and talking about his designs. He’s doing some great work on MyCamTools, and I’m looking forward to sharing the stage with him and Eli for Fearless JavaScript 2.0.

If you’re in Paris for the Sakai conference, we’d love to see you at the workshop. If you can’t make it, we’re hoping to create a podcast of the event. Here are the details:

Fearless JavaScript
Presented by Colin Clark, Eli Cochran, and Nicolaas Matthijs
Monday, June 30
1:30 - 4:30 pm in Amphi 55A

Topics will include:

* Why JavaScript?
* Fundamentals of JavaScript
* Using jQuery
* JavaScript in Sakai
* Portal and mash-up friendliness
* Accessibility

Everyone is welcome to attend the workshop. It’s assumed that you have some understanding of at least one programming language, and that you’ll bring a laptop to follow along with the hands-on exercises.

See you there!

“Premature optimization is the root of all evil.”

Antranig’s performance metrics are intended to help establish a gut feel about the cost of a particular operation. They’re low-level measurements, and as such they can’t take into account a fundamental variable: context. These performance metrics have very different consequences depending on the size, scale, and nature of the application. His examples are apt: intensive, highly recursive, up-front operations like template rendering show the full cost of JavaScript’s weak performance. Indeed, Antranig has used his research to good effect on our nascent client-side template renderer. In a relatively short amount of time, he was able to improve the performance of his low-level parsing algorithms by several orders of magnitude using regular expressions and fast string comparisons. Impressive stuff.

On the other hand, context matters a lot when it comes to user-facing behaviour, where timings may vary significantly based on the user’s choices. Many UI components, such as the Uploader, provide a fairly modal experience for the user. Such components undoubtedly have less stringent performance requirements, and our acceptance criteria should be determined by testing in real-world situations.

The risk with out of context performance measurements is the tendency to optimize prematurely and without accurate data. There’s only so much time in the day. Don’t lose the opportunity to add useful features to your application by starting in on time-intensive performance optimizations too early. Without good data, your hard work may only have a tiny impact on the overall perception of speediness. Worse yet, premature optimization can lead towards architectures that are poorly factored, redundant, and hard to maintain.

So how do we ensure good performance without compromising architectural quality?

1. Write code with good design principles in mind
2. Continually measure performance in real world situations
3. Refine code incrementally to improve performance

Let’s take jQuery as an example. If you look at the first release of jQuery, you’ll find that it was mostly composed of small functions that are clear and easy to read. jQuery’s author, John Resig, has always done a very good job of carefully profiling his code with performance tests, and over time the code base has evolved in certain areas to be faster and, as a concequence, occasionally less readable. He took the time to build something real first, then he optimized strategically. This is exactly the example of how we should all handle performance optimizations: build it cleanly, then identify areas where performance optimizations are necessary. Optimization without real-world numbers is, well, evil.

Building on Antranig’s advice, here are some strategies to help ensure both good performance and a maintainable design in JavaScript:

Avoid deep inheritance hierarchies

Excess use of inheritance tends to be brittle, hard to reuse, and slow. In JavaScript, inheritance is prototypal: an object has a secret prototype link that points to its parent object. When a property isn’t found on an instance directly, the object’s prototype is searched, then the prototype’s prototype, and onwards up the inheritance chain. This can be extremely slow on objects with a long lineage. JavaScript provides more effective strategies for reuse, such as containment and dynamic mix-ins. You’re better off with a very shallow inheritance hierarchy, using these techniques to share code across objects instead.

Break code into small, independent parts

JavaScript is a functional language, and functions serve as the fundamental building block for all programs. Break complicated algorithms up into smaller functions that take only the data necessary to perform a given operation. Small functions are more easily tested, modified, and reused. What about the cost per function call that Antranig mentions? If you need to optimize for speed later, you can easily merge several functions back into one. Analyze frequent code paths and identify specific bottlenecks first, then selectively fold smaller functions into a larger one.

Don’t deeply nest functions

In JavaScript, functions can be nested inside other functions. This is a very powerful technique, since inner functions have access to the variables and scope of the outer function even after the outer function has finished executing. This is called closure, and it’s one of the best ways to encapsulate state and behaviour in JavaScript. The trick is to avoid deep nesting; don’t stick a function inside a function inside a function. Similar to the inheritance chain, each JavaScript function invocation brings with it a scope chain, providing access to variables defined in outer functions. If a reference to a particular variable isn’t found inside a function’s scope, the JavaScript runtime will climb the scope chain, searching each outer scope in turn to find the property. All this searching can get pretty slow. Limit yourself to a couple of layers of closure at the most. Here’s a slight contrived code example and an illustration of the scope chain in action:

Illustrated Code Example: Using a closure for event handling

  function showMessage(messageToDisplay) {
  	var todaysPie = "Banana creme pie";

	return function(event) {
		alert(messageToDisplay + " " + todaysPie);
		showPictureOfPie(event.target, todaysPie);
	}
  }

  var messageClickHandler = showMessage("Welcome to my pie shop. Today's pie is:")
  $(element).click(clickHandler);

Identify nameable units of behaviour and encapsulate them

On the client side, it’s really easy to let your application logic get all muddled up with your presentation code. Large, hard to maintain functions are often structured like this: 1) Perform some calculations; 2) fetch some data; 3) update the DOM. These are separate tasks, and your code will be significantly easier to read and reuse if you split these behaviours into independent units. The trick is to look at your code while you’re writing it and recognize “things” that are easily identified and named. Using a closure or an object, you can factor out pieces of the overall behaviour and data into a separate entity. Here’s an example of the various units of behaviour we identified within the Reorderer:

• The Reorderer is responsible for binding events to DOM elements and providing end-user APIs
• A Layout provides an awareness of visual layout and spatial characteristics: grids, lists, portlets within columns, and so on
• The Mover is responsible for moving elements around in the DOM, deferring to the Layout for spatial information
• A Permissions object defines the rules for valid drop targets and operations on a specific element
• Keyboard selections and accessibility is provided by the jQuery keyboard-a11y plugin
• MouseDragAndDrop encapsulates our use of jQuery UI for low-level drag and drop functionality

Use closures to encapsulate data and behaviour

Antranig covered this one really well in his post. Under many circumstances, you’ll pay a significantly lower cost to access data wrapped in a closure than within an object, and functional idioms tend to be more flexible and easier to maintain than object-oriented ones. This is particularly true in JavaScript, where there is no built-in support for namespaces or privacy. Just be careful not to nest your closures too deeply.

Never, ever cut and paste code

It almost feels silly to repeat this one here. One of the foundational principles of programming, regardless of the language you’re using, is DRY: Don’t Repeat Yourself. The bottom line is this: cut and pasted code will come back to bite you. Don’t do it. Antranig’s post hints at the idea that it’s okay to cut and past code in JavaScript. For all but the most extreme cases, I don’t buy it. If you find yourself tempted to cut and paste something, take the time to factor it into a function. It will be easier to update later, and much easier to test. Duplicate code hurts!

What I mean here is to know, generally within a factor of 2 or 3, how much one operation is likely to cost relative to another one, and to have a similar kind of “fuzz factor” with respect to actual performance on a particular kind of machine you are familiar with. Back in the days when I was starting on RSF, pinned to my desk was the original “How Long, Oh Lord?” which I reproduce here:

For reference, these timings were on a Windows JDK 1.4 running on a relatively weak machine in 2005, perhaps something like a c.2Ghz P4.

Anyway, to skip to the present, starting work on the RSF/Fluid client-side renderer, I felt similarly at sea on how to plan for a design that runs efficiently. Javascript is starting to emerge from its “ghetto folkloric” period and to become the target of determined engineering efforts, and there are a number of resources round the web focused on Javascript performance issues, such as John Resig’s excellent blog entry collection http://ejohn.org/blog/tags/performance/, and a splendid presentation by Julien Lecomte of YUI http://yuiblog.com/blog/2007/12/20/video-lecomte/.

Some of the “learnings” I report here in “How Long, Oh Lord 2008″ are part of Julien’s presentation, but others I have not seen before. The workhorse of these estimates is a microbenchmark collection I have put together at https://source.caret.cam.ac.uk/rsf/projects/RSFSamples/trunk/AjaxJSON/src/webapp/content/templates/perftest.html (this is just accidentally for now embedded in the middle of a wider Java/RSF/JSON/Maven development structure).

The benchmarks are run in (currently 5) “batches” of say 10,000 iterations, the most extreme measurements are discarded, and the average of the remaining three batches are reported as a headline figure. This is an attempt to defeat random variation caused by system business, GC collections and the like.

Here is a sample row and table output from the system:

Function Calls Considered Harmful

I think the most central result I want to report here is the truly fantastic expense of function calls in Javascript, which to my mind generally cuts at the heart of the possibility of orderly development.

The bad news is that a simple function call costs around 3 microseconds on a typical machine in Firefox 2.x. Opera builds are generally much better, around the 1µs mark. The “better” news going forwards is that current builds of Firefox 3 betas are vastly improved, down to 350ns on the same hardware. All the same, the ground reality is that in the vast majority of installed system for a number of years, a Javascript function call is going to cost something in the region of 1 microsecond. Let’s put that figure into perspective with a tour of our glorious history - the heroic IBM System/360 Model 30 (Announced April 7, 1964) had a machine cycle time of 1 microsecond. On the personal computer front, the Commodore PET (January 1977) was the same. In this respect (and some others) therefore Javascript sets back computer science 30 years or more. This is a little tongue-in-cheek - but not entirely. Clearly even on the most up-to-date environment, a function call in Java, say, will not execute in a single machine cycle. But the discrepancy is more like 1 order of magnitude, rather than 3 as it is in Javascript.

Why is this, and what does it mean?

Javascript function calls, as the most skilled practitioners of Javascript (Crockford, etc.) will explain, are vastly powerful, vastly underappreciated pieces of machinery. Every invocation of a Javascript function brings into existence a highly complex structure, a dynamic scope chain which will allow variables to be referenced long after a function has returned — or else, require them to be garbage collected, piece by piece. Therefore a JS “stack frame” is really very much more like an “object” in a classic “OO” language in its capabilities. More on this later. This means that a truly efficient implementation of JS function calls is essentially impossible - although the Firefox 3 team are having a thoroughly good try. What is particularly happy about this situation is that, as Crockford frequently laments, the true power of Javascript functional expressiveness is lost on 99% of its practitioners, who are typically trained if at all) in far less mind-bending idioms. And for those 1% who do truly understand what a JS function call could do, how many can make more than a handful of the function calls they write count for their full power?

What does this mean? It means that truly performant Javascript code is really obliged to drop back more than decades in its structural idiom. The function call is the primary, the first and most useful, unit of modular abstraction in programming, and these costs imply that efficient Javascript code must degrade to consist of few, extremely large function bodies, whereas a design based on fine-grained, flexibly composed units would be much more comprehensible and easy to maintain.

The most skilled Javascript programmers, framework authors, have been aware of this (implicitly or explicitly) for a long while, and it is clear that their code is following suit. As one example, TrimPath Templates a very performant macro-style templating language, is implemented with a large work-horse method (actually TrimPath uses a number of other performance tricks which will be touched upon later). As another, JQuery’s central workhorse, JQuery.find() is likewise a single monster method. In this and many other areas, “stupider is better” is often the correct answer - cut and paste, traditionally the anathema of programming, has a new lease of life in Javascript as it is frequently superior to simply repeat frequently occurring, especially small, snippets of code, rather than factor them into function calls.

Scoping and storing bugbears

The pattern of which operations in Javascript are performant and which are not is interestingly skew. To go into other results from “perftest”, we can discover a rule of thumb that one object property access is roughly equivalent to four string comparisons. For example, when testing a string value to be one of four possible values, looking it up as an Object member is roughly breaking even with writing out the sequential comparisons.

Similarly, access to objects in outer scopes carries a price. For example, again on FF2.x, simply accessing a variable not declared at the current function scope carries a penalty of 300ns. Vast, in “traditional” terms.

Facts like this militate for numerous “hoisting” tricks - essentially if a given expression or variable is accessed more than a single time in a particular function scope, it will almost always make sense to assign it to a local variable before use. This is obviously much more important for Object members than for “outer scope” variables. Julien Lecomte covers this in his presentation - a further comparison worth noticing is that function scopes are actually more more efficient containers for data than Objects. This further pushes forwards the Crockfordian view of “function scopes” as being the primary units of abstraction and storage in an “idiomatic Javascript” program, rather than traditional “Objects”. If one is going to pay 1 microsecond for a function call, better make it count - but better to pay 3 microsecond for a call and 300 ns for an access, than to pay 5 microseconds to construct an Object and then 500 ns per access (Firefox 2.x figures - FF3 and Opera are better, but the relative picture is unchanged).

An unfortunate discovery

On Firefox 2.x the already dim picture is muddied further by the discovery that simply invoking a function declared at another function scope nearly doubles the cost - even if no variables are declared or accessed at the outer scope. Since this “trick” is the now standard modern recommdation for implementing namespacing (Crockford on Privacy) this automatically penalises those who seek to organise their code better. Again blissfully this penalty is gone from Firefox 3.

Finding performant primitives

Some Javascript operations essentially retain near-native performance. Boolean tests, numeric operations, local control structures such as loops and branches, all proceed pretty quickly. This would seem to at least allow the possibility of some high-performing algorithms (assuming they were structured as giant function blocks) were it not for another really unfortunate Javascript feature - the lack of any bulk “primitive” storage. Java’s primitive array support was often widely derided as a really irritating and unorthogonal language feature, but actually is the only thing that saves it as a data processing language. In Javascript one has only Strings - and even a simple charAt() call causes a complete object instantiation (as a side tip - use of the less-well-known String.charCodeAt() followed by a (much less readable) numeric test is around 20% faster than a charAt() plus String test). There is simply nowhere to put extensive data since every Array is an Array of Object.

This leaves one to fall back on Strings and String tricks. An interesting discovery is that String.indexOf() is a startlingly rapid operation - after what appears to be a standard “native call overhead” of something like 1.5 microseconds (abouthalf a “standard function call”) (again FF 2.x), the actual search occurs at fully native speeds. This implies you can find a given String amongst around 10K of data practically as fast as you can find it amongst 100 bytes. TrimPath makes very fruitful use of indexOf to gain a very good speed boost, helped by its simple and “easily recognisable” templating format.

Regexes can take up a lot of slack in some cases, but their overhead is very much higher than an indexOf. As Lecomte notes, a Regexp.test() is *somewhat* faster than a Regexp.exec() but actually not startlingly so, since the test() appears to require no “user-visible” memory allocation. Any Regexp invocation carries a standard overhead of about 2 function calls, so use with care on simple tasks.

Garbage in, garbage out

Investigating the costs of garbage in Javascript rounds out most of this discussion. It is very interesting to repeat the performance tests on a “huge browser image” runtime, since this immediately exposes the kinds of operations which create garbage and those which do not. I am somewhat untypical in that I am frequently running a system with around 70 top-level Opera frames (it is the only browser that will cope with this sensibly) and perhaps 20 Firefox frames. In a browser image like this, the fact that all Javascript objects are typically thrown into a global shared heap, rather than the per-frame heaps one might perhaps wish for is painfully exposed. Running perftest.js in a “mega-Opera” shows the allocating tests running perhaps 100 times slower - the data becomes very spiky, but a single micro-test can end up taking a full millisecond rather than a microsecond.
This exposes that although function calls are “like” Object (heap) allocations, they are still underlyingly dealt with by different, and more efficient data structures - even in a “mega-Opera”, a bare function call is still taking 1 microsecond, whilst a true heap allocation is running at a millisecond or more. “mega-Opera” is a highly untypical environment, but it demonstrates at extremes the superior scaling costs of function frames over true garbage.

If there is a simple take-home message from all of this, it is -

i) Javascript function calls are hugely expensive compared to traditional environments, take great care not to factor your code too deeply,

ii) Javascript function calls are sufficiently powerful to take over many of the traditional roles of “Objects” in traditional environments, so consider designs that operate closure-based storage and reserve your true Objects for data transfer,

iii) make judicious use of the primitive operations which are provided (indexOf and regexps) where possible to make your code run reasonably.

One of the things we’ve needed to improve in the Reorderer is the API. In earlier versions, it was pretty hard to initialize your own Reorderer. You needed to know a lot about the inner workings of objects such as LayoutHandlers, item finders, and so on. Infusion 0.3 solves this problem by providing clean, one-line functions for creating your favourite configurations of the Reorderer:

Sorting Lists

fluid.reorderList(containerSelector, itemSelector, orderChangedCallback, options);

Pass in a selector string to locate your list and the items within it, and you’ve got yourself an accessible, sortable list. Optionally, you can also specify a callback function that will be invoked every time the list order changes, along with a whole raft of other options for configuring the Reorderer.

Sorting Grids

fluid.reorderGrid(containerSelector, itemSelector, orderChangedCallback, options);

Same thing. Just pass in a selector string to locate your grid and the items within it, along with an optional order changed callback and some configuration options. Boom, a sortable 2D grid.

Sorting Layouts

fluid.reorderLayout(containerSelector, layoutSelectors, orderChangedCallback, options);

This one is for reordering portlets, content blocks, or other chunks of layout on a page. Pass in a selector to locate the container of your layout. Pass in an object that contains two selectors: one for the columns in the layout and one for the portlets. This function will do all the heavy lifting for you, figuring out which portlets are in each column, building a Layout object, and so on.

If you want all the advanced portal features of the Layout Customizer, such as locking, JSON-based layout, and permissions, you can always drop down to the lower-level API:

fluid.initLayoutCustomizer(layout, permissions, orderChangedURL, options);

This function exposes some of the more complex objects used by the Layout Customizer, the Layout and Permissions objects. The Layout is a JSON-style object representing the column and portlet structure in a DOM-agnostic way. Permissions is a complex data structure encoding all of the valid drop target permissions for each portlet and column in the layout. Generally, you’ll want to use the simpler fluid.reorderLayout() function unless you need the advanced portal features.

Documentation

We realize that code examples and the occasional blog post aren’t nearly enough to get you started using Infusion. Anastasia has been experimenting with some great new documentation for the 0.3 release. It’s still very much a work in progress, but check out her live examples of how to use the Reorderer. She’s still figuring out the right recipe for our documentation, so please share your ideas and thoughts with her!

“Jess comes with an incredibly rich and relevant background in managing large, complex, distributed projects. She is very familiar with the academic environment, the schools engaged in Fluid and the open source and community source development process. Until recently, Jess was the Innovative Projects Manager at Duke University and managed, among many innovative projects, the Duke Digital Initiative.”

I’m impressed with how diverse Jess’ background is. She has a firm grasp of technical issues and was a volunteer with the Geekcorps project to build independent network infrastructure in Ghana. She’s managed a variety of projects in a higher ed setting, and has taught project courses at Duke University. Her academic background in Political Science and Philosophy brings a unique perspective on community building and open source culture. And I know she’s been learning a ton about inclusive design, too!

A quick bit of history: Fluid has been trying to hire a dedicated Project Manager for over a year now. Hiring at a university can be pretty slow and bureaucratic, and we had several promising candidates who were unexpectedly wooed away by offers from the private sector. Throughout this process, we took the time to be sure that we hired someone who will thrive within the unique constraints of a distributed, community-driven environment. I’m thoroughly confident that Jess will excel in this role, and she is committed to helping us communicate and collaborate better as our community grows.

On a personal level, I’m relieved by the prospect of being able to focus on technical leadership for Fluid. With all the task-juggling and cat-herding I’ve done over the past year and a half, it’s been hard to put enough attention towards architecture and coding. Now that Jess is on board, I’ll be much more focused on development work. I have a backlog of technical writing and coding to catch up with, so expect to hear a lot more from me on the lists and this blog with code examples, screenshots of new components, and more.

Jess will be based in Boston, but is planning to do a fair bit of traveling over the next few months. She’s also really interested in hearing from members of the community, so don’t hesitate to drop her a line. Please join me in welcoming Jess to the Fluid community!

Why Use a JavaScript Toolkit at All?

Quite simply: browser incompatibilities. There are other reasons to use a toolkit, but the foremost among these is the need to work around inconsistencies and bugs across browsers. Despite attempts at defining standards for JavaScript and the Document Object Model (DOM), there are still major differences in the way browsers handle basic constructs such as events, AJAX connections, and style sheets.

Anyone who writes a lot of JavaScript code without a toolkit will end up with an accumulation of workarounds and object detection checks that will eventually amount to a home-grown toolkit. Writing your own cross-browser code usually starts out easily enough, but eventually bogs down in a maze of subtle and obscure workarounds. Most good JavaScript programmers have internalized these incompatibilities over time, but this knowledge is painfully acquired through time-consuming, frustrating debugging sessions. Take a look at Quirks Mode for all the gory details about browser incompatibilities. The bottom line for me: it’s crazy not to use a JavaScript toolkit.

The decision boils down to cost. Do I want to pay for the all development, maintenance, and testing of an entire JavaScript toolkit? Or do I want to share that cost, and the hard expertise that goes into it, with thousands of other users of a toolkit like jQuery? For Fluid, our priority is to build great user interfaces, not to bang our heads against browser incompatibilities all day. So we knew we had to choose a toolkit.

Evaluating Toolkits

Picking a JavaScript toolkit can be pretty bewildering. The Web 2.0 scene is littered with options, each with very different features, philosophies, and licenses. It’s analogous to the mess we have on the server side with Java presentation frameworks. Superficially, a wide variety of choice seems like a good thing. But practically, we have a plethora of overlapping, inconsistent options that largely do the same thing. To further complicate matters, the client-side UI world is still in its infancy, so this degree of choice and fragmentation ends up leading to a lot of bugs and incomplete features.

Early on in the project, we came up with a set of criteria for evaluating DHTML toolkits. These included:

• Cross-browser support
• Easy debugging
• Event abstraction
• A solid DOM manipulation library
• A strong community and clear roadmap for improvements
• Accessibility

Accessibility is a particularly challenging issue for toolkits, opening up a whole new can of worms for Web developers. The ugly reality is that the vast majority of widgets are completely inaccessible to users who can’t use the mouse or who require an assistive technology such as a screen reader. Worse yet, the standards and techniques for DHTML accessibility are still being worked out, and browser support is significantly in flux.

We’ve been involved in the Dojo community for over a year now, assisting with their efforts to make many of the Dijit widgets more accessible. This work includes keyboard navigation, ARIA semantics, and high-contrast styling. The whole team has done a first-rate job on Dijit accessibility, and continue to make excellent progress.

Many of you might remember our initial efforts to use Dojo as the toolkit of choice for Fluid. We built the Reorderer as a Dijit Widget, and used their drag and drop library extensively. Problems ensued. Some were entirely situational; the Dojo community was pushing hard for their 1.0 release and weren’t able to give as much attention to their drag and drop library as we would have liked. Others were architectural; our unobtrusive, markup-driven approach just didn’t fit well with Dojo’s more “JSF-ish” style of widget encapsulation. We genuinely hoped to leverage Dojo for Fluid, but ultimately realized that it just wasn’t aligned with our needs.

With a broken Reorderer and a release deadline looming, we decided to switch. In a week-long sprint, we implemented the Reorderer’s core using YUI, Ext, jQuery, and MochiKit. We were never able to do a full “apples-to-apples” comparison, but we got a pretty good feeling for each toolkit’s various strengths and weaknesses even though we had to choose quickly. YUI’s documentation was exceptionally thorough, and its feature set was excellent. MochiKit provided a nice clean library. Dojo widgets had great built-in accessibility. But ultimately, we chose jQuery because it just fit.

jQuery Just Fits

jQuery is a pretty remarkable API to work with because it was built with the skills of Web developers in mind. It leverages familiar Web design/development techniques such as CSS selectors, so it’s easy to apply the things you already know and get started with it quickly. John Resig, jQuery’s creator, frequently describes how non-coders are attracted to jQuery because it fits with their “user mental model” as a Web developer. And at the same time, it seamlessly encourages the use of good practices such as unobtrusive JavaScript and functional programming.

jQuery has a very small and focussed feature set. I’ll spare you the usual byte-for-byte comparisons, but it is tiny and carefully organized. jQuery is designed specifically to make the common, tedious tasks of DHTML development faster, easier, and more succinct. If you’ve done any JavaScript programming, you’ll know that the vast majority of time is spent finding things in the document and doing stuff with them. Recognizing this, jQuery optimizes the process of searching for elements in the DOM and performing repetitive actions on them. With jQuery, binding event handlers or applying styling to a set of elements becomes a trivial, one-line exercise without the need for verbose loops. A code example is probably the best way to illustrate this point.

Unlike other toolkits, jQuery doesn’t bother with features such as class-based inheritance or other abstractions on top of JavaScript, and the end result is much cleaner and more comprehensible. After the switch to jQuery, our code base got much smaller. Its selectors and chainability encourage terseness in a good way; code that is minimal but clear and well-named.

I also like the fact that jQuery specifically makes it easier to keep your code and markup separate so that they can be changed independently. The browser environment makes separating logic from presentation feasible, and jQuery encourages this by providing quick ways to grab elements in the document and bind handlers to them when the page first loads. jQuery’s UI widgets are built in such a way that it’s easy to support progressive enhancement, allowing your pages to to be built on a bed of valid semantic markup which will degrade gracefully on older browsers, mobile devices, and so on. Here’s an example of how jQuery UI Tabs take advantage of progressive enhancement.

For features that don’t readily fit within jQuery’s compact core distribution, a “plugin” architecture is provided. I’m using quotes here, since jQuery plugins are written using a simple set of techniques that are based on JavaScript’s prototypal inheritance. There’s no special API here; if you already know how to extend objects in JavaScript, you’ll find jQuery plugins simple to write. A rich set of programming utilities and user interfaces have been released as jQuery plugins by members of the community. I still have reservations about whether the jQuery plugin model is workable in all scenarios. I expect that there will be types of services and components that don’t naturally fit within the jQuery selection-based model.

I mentioned the community earlier. The jQuery lists are healthy and active, and there’s a general sense of openness and welcoming from the group. I gather that, in the early days, John Resig took a lot of time out of his day to patiently answer questions, and this has clearly fostered a generous user culture.

How You Can Benefit From Our Commitment to jQuery

We think that a healthy, open Web should include more than one accessible DHTML toolkit. The Mozilla Foundation agrees with us, and have kindly offered us some funding to help mentor the jQuery community on accessibility issues. We’ll help ensure that the jQuery widget set is accessible, and that there are simple tools available to make third-party jQuery plugins accessible.

Fluid is contributing a lot of the code required by DHTML developers to support accessibility in their own widgets and plugins. And other members of the jQuery community are getting involved. We expect a rich set of accessibility and progressive enhancement tools to emerge over the coming months. Many of these accessibility plugins are ready to use now, and we’d love your feedback and suggestions for how we can improve them in the future.

If you’re already using jQuery, Fluid’s components will merge seamlessly with your existing code. Over time, we’re planning to build jQuery plugin interfaces for many of our components, providing a quick and idiomatic way of using Fluid components in your pages.

Summing Up

There’s a lot of exciting action going on in the jQuery and Fluid communities these days. The jQuery UI team is on the cusp of releasing jQuery UI 1.5, which will provide a significant reworking of the widget model and opens the door for accessibility code to be easily shared across widgets. Fluid’s upcoming Infusion 0.3 release is entirely based on jQuery, and we’re already using a pre-release version of their 1.5 drag and drop library with great success. Expect to see upcoming Fluid components incorporate other jQuery UI widgets, and we’ll contributing usability and accessibility advice back to the community.

We recognize that not everyone is going to make the same toolkit decisions we did, so we’ll do our best to ensure that Fluid’s code plays nice with other toolkits.

Oh, and if you live in Toronto and want to contribute to our work, we’re hiring…

What are uCamps?

uCamps are informal, collaborative events designed for the whole spectrum of contributors to successful software projects–developers, testers, designers, and everyone in between. The idea is to get everyone in a room to discuss UX tools and techniques, share our experience, and collaborate on real projects together.

One of the motivations for starting the uCamp was the belief that everyone should share a basic vocabulary for user experience, and that we should promote techniques that help everyone engage in making open source software better. uCamps were modeled after the DCamps organized by Rashmi Sinha and others in the Free/Open Source Software world. They’re intentionally ad-hoc and informal; bring your ideas, techniques, and inspiration and share them with the group. Our uCamps usually include lightning talks about various design-related topics, hands-on workshops, and time to work with your colleagues.

The uCamps are one of our longest-standing Fluid activities. The first uCamp was held in Atlanta back in 2006, when Fluid was still just in the proposal phase. A great group of volunteers have worked hard since then to ensure that the uCamps are a central point for UX collaboration within the Sakai and uPortal communities.

Sharing the Effort

As many of you know, Fluid is entering its second year as a project. We dedicated our first year to community building, and the uCamps are one example of our success in this regard. Fluid has become the place to go to talk about and collaborate on great user interfaces in the community source world. In our second year, we want to build on the strong community we’ve established and focus our efforts on creating new user interface components, design patterns, and the JavaScript framework to support them.

To this end, we’d like to pass the uCamp torch to other members of the community. The uCamp at the JA-SIG conference in St. Paul’s will be our last Fluid-funded uCamp for awhile, but we’d still like to help other motivated members of the community offer them in the future. If you’re interested in leading the next uCamp, we’ve got a great page of resources on how to organize a uCamp. Paul Zablosky, our uCamp Coordinator, is willing to help get you started. And of course, we’ll be still be here to help with lightning talks and participate in collaborative activities.

A special thanks to everyone who has helped with and participated in the uCamps over the past couple years. And don’t forget to join us in St. Paul’s!

Getting the Source Code

The source code is currently available in the Fluid Project SVN, and will be included in the Fluid 0.3 release. Long term, we hope to include this plugin directly in jQuery UI. jqUnit tests are also available.

This plugin works on FireFox, Safari, Internet Explorer 6+, and Opera 9.5+. If people find this plugin useful, I’d be happy to contribute it to the jQuery Plugin repository. I’ve licensed this code in the same manner as jQuery, so it’s available under a dual GPL/MIT license.

How to Use It

The API is super-simple and should be familiar to jQuery users:

jQuery().tabindex()
Gets the tabindex value of the first matched element. If the element doesn’t have a tabindex attribute, undefined is returned. The value returned is normalized to a Number, since browser implementations vary.

jQuery().tabindex(value)
Sets the tabindex value on all matched elements. value can be either a String or a Number, but valid tabindex attributes need to be integers.

jQuery().removeTabindex()
Removes the tabindex attribute from all matched elements.

jQuery().hasTabindex()
Checks for the presence of a tabindex attribute on the first matched element. Returns true if the element has a tabindex attribute, and returns false if tabindex is not set. Neither Simon or I have yet to come up with a reliable method for querying tabindex attributes on versions of IE < 6, so this method will always return false on IE 5.5 and below.

Future Directions

I wrote this tabindex plugin as part of a larger project to build a comprehensive keyboard accessibility plugin for jQuery. While it’s not quite finished, this plugin will make it easy for developers to add keyboard handlers to their code without a lot of extra overhead. A common problem in the DHTML world is that user interfaces are built to work solely with the mouse, excluding users who need to use the keyboard to navigate. jquery.keyboard-a11y takes the hard work out of making your interfaces usable with the tab and arrow keys.

This is the first library that I know of that provides DHTML keyboard accessibility without requiring you to implement a whole model for widgets. While it doesn’t solve all your accessibility needs, it’s designed to work with lots of different types of markup and be consistent with the jQuery approach. Here’s a quick summary of the API:

jQuery().tabbable()
Adds all matched elements to the tab order by giving them a tabindex attribute of “0.”

jQuery().selectable(container, customHandlers, options)
Makes all matched elements selectable with the arrow keys. Provide your own customHandlers to inject custom styling and logic. Options lets you modify the plugin’s default selection behaviour.

jQuery().activatable(customHandlers, options)
Makes all matched elements activatable with the Spacebar and Enter keys. Provide your own handlers to trigger custom behaviour. Options allow you to activate elements using other keystrokes.

Source Code

The in-progress code for this jQuery keyboard accessibility plugin is freely available. There’s still a lot of work be done before it will be ready for prime time, but I’d encourage you to take a look and let me know what you think.

jquery.keyboard-a11y.js

Summary

We’ve been really excited about using jQuery on the Fluid Project. Our Reorderer, an accessible library for drag-and-drop sorting of elements, is built entirely on jQuery and jQuery UI. We hope these new accessibility plugins will be helpful in enabling greater DHTML accessibility within the jQuery and Fluid communities.

I’d love to hear your feedback or suggestions for these plugins. Don’t hesitate to get in touch.

Browsers tested

• Internet Explorer 6 on Windows XP
• Internet Explorer 7 on Windows XP
• Firefox 2 on Windows XP
• Firefox 3 beta3pre (the Minefield nightly build of 2008-01-07) on Windows XP
• Safari 3.0.4 on Mac OS X 10.5.1
• Opera 9.25 on Windows XP

Getting the tabindex value from an element

Element.tabIndex and Element.getAttribute("tabindex") can be used to get the tabindex value on both HTML and XHTML documents:

Element.getAttribute("tabIndex") can be used to get the value on HTML documents. However, Firefox, Safari, and Opera treat XHTML documents case-sensitively and, when used on an XHTML document, Element.getAttribute("tabIndex") will return null.

Getting the tabindex value on an element that doesn’t have one explicitly set

On elements allowed to have tabindex attributes in HTML4, the tabIndex property defaults to 0 when no tabindex has been explicitly set:

On elements not allowed to have tabindex attributes in HTML4 (but supported by IE and Firefox and the upcoming HTML5), the tabIndex property defaults to 0 on Internet Explorer, -1 on Firefox, and undefined on Safari 3 and Opera 9:

On Internet Explorer, Element.getAttribute("tabindex") returns 0 on elements that have no tabindex value explicitly set. The other browsers return null:

Testing if tabindex has been set on an element

On Internet Explorer we can’t use Element.getAttribute("tabindex") to determine if a tabindex has been specified on an element because it returns 0 even when one hasn’t been. IE also doesn’t implement Element.hasAttribute(). However, IE6 and IE7 do implement Element.getAttributeNode() and we can use that (Element.getAttributeNode() is also available on Firefox, Safari, and Opera.) When it is called on Internet Explorer on an element with no tabindex value set, Element.getAttributeNode() will return a node whose specified property is false. When called on Firefox, Safari, and Opera it will return null. For cross-browser compatibility, we can use a test like this:

var attr = elem.getAttributeNode("tabindex");
return attr ? attr.specified : false;

Setting the tabindex value on an element

Element.setAttribute("tabIndex") must be used on Internet Explorer. Either “tabindex” or “tabIndex” can be used for HTML on Firefox, Safari, and Opera, but “tabindex” must be used for XHTML:

Removing the tabindex value from an element

Like setAttribute(), Element.removeAttribute(”tabIndex”) must be used on Internet Explorer and either “tabindex” or “tabIndex” can be used for HTML on Firefox, Safari, and Opera. “tabindex” must be used for XHTML:

Test files, detailed results, and Dojo implementation

• HTML test file for getting the tabindex value
• XHTML test file for getting the tabindex value
• Results for getting the tabindex
• HTML test file for setting and removing the tabindex
• XHTML test file for setting and removing the tabindex
• Results for setting and removing the tabindex
• Dojo attribute functions (search for “Element attribute Functions”)
• Unit tests for Dojo attribute functions

Updated 2008-01-10: corrected removeAttribute(”tabIndex”) results

For those of you who are interested in Sakai, if you haven’t already read Michael Korcuska’s blog entry about his impressions of the summit, I’d encourage you to take a look. He’s done a great job of outlining his perspectives on our work and Fluid’s role within the Sakai community.

We also had strong representation from the uPortal community, including Eric Dalquist from UW-Madison, Paul and Jens from UBC, and Jennifery Bourey and Susan Bramhall via videoconference from Yale.

Moodle was represented by the crew from York, who brought with them interesting results from their recent UX Walkthroughs as well as a sneak peek of the new Open VULab remote testing product. Keep an eye out for more information from Ron and his team about the VULab in the coming weeks.

It’s been a busy two weeks with lots of project coordination tasks arising from the summit, but here’s my long-awaited summary of the week:

Project Vision and Roadmap

As a whole group, we discussed the project release plan and vision. The release plan outlines the timing and deliverables for each of our quarterly release milestones, starting with a 0.1 version of the Fluid technology and UX Toolkit later this month, and culminating in a 1.0 release in March 2009. More detail needs to be fleshed out in this document based on the results coming out of our design working groups, but I think it outlines the overall structure and deliverables of the projects quite well.

The vision statement is a work in progress. In my mind, the challenge of writing a good project vision is to capture the goals and philosophy of the project at a high level, while still managing to keep it succinct and focussed. I’ll be revising this document in the coming weeks, and I’d appreciate any feedback you may have.

Architecture

The technical group discussed the latest overview of the Fluid architecture plan. One of my goals in revising the architecture overview was to provide more information about the current reality of the Fluid framework and a clearer picture of how we think the framework will evolve as we build new and more complex components. Many of the technical discussions at the summit centered around the requirements for a contract between client and server. In time for the 0.1 release, we’ll be creating comprehensive documentation about the contract of the Reorderer and other Fluid components within the wiki.

We also talked a lot about how a Fluid component is defined from the technical perspective. In short, a component is a reusable, embeddable chunk of user interface can be adapted to suit different contexts. A Fluid component is more than just a widget, embodying a larger workflow or activity within the system that may cut across multiple tools or portlets.

Technically-speaking, Fluid components are built using JavaScript, HTML, and CSS running on the client. Component logic is composed from smaller units of functionality, providing a built-in model for flexibility and adaptation. Layout handlers, keyboard bindings, and other UI behaviour can be extended or customized by setting properties of the component. A component collaborates with service logic on the server, including the request handling and markup generation capability of a server-side presentation framework.

I’ve pared back some of the fuzzier aspects of the architecture, including the component container, since we haven’t yet seen a need for strict life cycle control over components. I suspect that as we build up larger components with a lot of configurable properties, we may reintroduce the concept of a component container to handle some inversion of control functionality on the client. This will help make it easier to build a component out of parameterizable chunks of behaviour and wire them up at runtime. In the meantime, our approach to using standard markup and RESTful server-side conventions provides sufficient support for pluggable, reusable UI components without need for the container.

Client-Side JavaScript Toolkits

Joseph Scheuhammer presented an analysis of JavaScript toolkits, outlining the issues to consider when selecting a toolkit for Fluid. I think this is a very insightful analysis and is worth taking a look at.

Simon Bates presented the accessibility work his team here at the ATRC has been doing on the Dojo Toolkit. It was clear that the work invested in Dojo accessibility has the potential to provide a substantial benefit to Fluid, allowing us to adopt and reuse the Dijit widget set without having to write a lot of extra code to make them accessible. Dojo is the only toolkit to date that has made a commitment to built-in accessibility. On the other hand, the technical group at the summit raised some valid concerns about Dojo’s architectural approach, particularly in light of our desire to leverage server-generated markup where appropriate.

Ultimately, the technical team at the summit agreed to bring a draft proposal to the community recommending the use of Dojo for building Fluid components. We’re still free to use non-Dojo widgets where necessary, but there will be a greater responsibility to ensure consistency and accessibility if we do so. With this recommendation comes a commitment to get more involved in the Dojo community and help to support and influence its direction.

I’ll send out a separate email documenting this proposal soon, and we’d like lots of feedback from the community about this decision. Given the lingering concerns about Dojo’s fit with Fluid’s architecture, we’ll revisit this recommendation over the next few months to assess its success.

Server-Side Presentation Frameworks

The technical team at the summit chose not to recommend a particular server-side presentation framework at this time. Instead, our approach is to incrementally define a set of requirements that any server-side presentation framework may conform to. We’ll build these requirements up as we create more examples of Fluid components and refine their communication with the server. Ultimately this will lead to a thin binding layer between client and server that we will implement ourselves in at least RSFand uPortal’s XML-based presentation pipeline. Spring Web/Portlet MVC is being considered as well. Users of other presentation frameworks will be able to implement this binding layer in their technology of choice, and are encouraged to do so.

During the technical sessions, we all recognized the hard work that Antranig Basman and the RSF team have invested in making RSF excel at delivering the markup and bindings required to enable cross-cutting user interface components on the server-side. If you’re working on a new tool or portlet, I’d suggest giving RSF serious consideration. I think there is going to be a lot of complimentary back-and-forth between the Fluid framework and RSF, particularly around the Universal View Bus, RSF’s strategy for client-side messaging.

Coding Session and Technical Working Groups

On Thursday, the technical group broke up into smaller teams to do some coding and flesh out a few key areas for the Fluid architecture. Eric, Anastasia Cheetham, Joseph, and Antranig all worked on integrating the Reorderer with uPortal’s drag and drop layout preferences. This will provide a more accessible solution for directly rearranging portals, tabs, and columns, and is expected to be complete in time for the uPortal 3 release in December. The technical solution cooked up by this team is really quite remarkable, and you can find more discussion of their work on the uPortal-dev and fluid-work lists.

Aaron Zeckoski and Michelle D’Souza worked on a new accessible AutoComplete component based on the Dijit ComboBox widget. While in itself this is a very small component by Fluid standards, Aaron and Michelle started in on the coding work required to make all Dijit widgets bind seamlessly with RSF.

I missed out on most of the coding session, but had an opportunity instead to discuss the challenges of user experience development in open source communities with Sakai’s new ED, Michael Korcuska, and a few of the Sakai board members who were at the summit. I’m really impressed with Michael’s work so far, and am looking forward to Fluid’s involvement in the new Sakai UX initiative.

User Experience Working Groups

One of my favourite things about leading the Fluid Project is the opportunity to be involved in the design process with our UX experts. Unfortunately at the summit I wasn’t able to spend as much time in the UX meetings as I would have liked, but the team has posted a great summary of the topics they discussed and their planned next steps.

During the week, the UX group worked extremely hard to sift through a lot of raw data collected during recent UX Walkthroughs. They identified a number of problem areas that are common across all our participating projects, and prioritized them based on both the frequency and severity of the problems. Based on this, the UX group has formed a number of smaller design teams that will delve deeper into specific areas and start designing solutions for the highest-priority issues. These teams will be creating new designs in the areas of Navigation, User Feedback, File Management and Workflow. It will take a bit more time before we start to see implementable components emerging from these working groups, but I’m enthusiastic about their progress to date. If you haven’t already, please consider getting involved in one of the groups and helping to design new Fluid components.

Wrapping Up

The most remarkable part of the summit for me was the combined sessions held with the whole group of 30 or so attendees. It was clear to me that we’ve built an effective community of contributors who are dedicated to fostering usability and accessibility within our projects. Over the five days of the summit, we worked through a vast array of topics, defining solid plans and next steps for all aspects of the project.

Now’s a great time to get involved in Fluid. Join a design team, contribute to the technical development, or lend a hand preparing for the upcoming 0.1 release. If you’d like to help out, or have questions about our work, please feel free to drop a note to the fluid-work mailing list or get in touch with me directly.