[dojo-contributors] Module formats for the browser

Kris Zyp kzyp at dojotoolkit.org
Thu Mar 25 11:14:41 EDT 2010


Reviving this old thread... There has been a lot of work lately in
CommonJS in improving asynchronous module loading, and I think it is now
at the point were it makes sense for Dojo to align with CommonJS modules
for future Dojo module loading. I know we have had some apprehensions
about the CommonJS module system, since it started with only a
synchronous mechanism, and we know the woes of such a system for the
browser (at least I didn't see it as being useful for Dojo). But module
transportation and asynchronous module loading specifications have
evolved since then, and I believe that CommonJS modules + CommonJS
transport [1] + RequireJS is the right path forward for Dojo.

More specifically this is what I would like to see us move towards:
Within Dojo source code, I would like to see us move to using the
standard CommonJS module format. This is the plain synchronous format.
Bear with me. For all development directly on these source files, we use
automated server side module transport wrapping that allows the files to
be loaded asynchronously. Our two distributions of Dojo will consist of
the uncompressed (instead of calling it the "src" dist), and compressed
version. Both of these distributions will have the Dojo modules packaged
with the transport wrapping.

Dojo users will then have two options for writing modules:
* Use RequireJS's async module definition API. Modules would look
something like:
require.def("my-module", ["dojo/date/locale"], function(locale){
  return {
    someFunction: function(){
       var formatted = locale.format(someDate);
        ...
    }
  };
});

This format does not require any server side wrapping.

* Use CommonJS format with a server side module wrapper. The source code
for a module would look like:
var locale = require("dojo/date/locale");
exports.someFunction = function(){
   var formatted = locale.format(someDate);
   ...
};

The server side module wrapper would output something like this to the
browser:
require.define({
 "my-module": function(require, exports){
   var locale = require("dojo/date/locale");
   exports.someFunction = function(){
     var formatted = locale.format(someDate);
      ...
   }
}, ["dojo/date/locale"]);

This CommonJS module standard format is what I am recommending that we
use for Dojo source code.

So why would we use the CommonJS module format if it necessitates server
side wrapping? I know there is extra burden for server side wrapping,
but I believe there are several key reasons this is advantageous:
* All Dojo modules can be CommonJS compliant modules, and easily be
reused on any CommonJS platform.
* There is less boilerplate that has to be written (and maintained) by
us as developers than the load-by-function approach.
* Modules are decoupled from their namespace, making them much more
portable.
Also, the server wrapping process is intentionally extremely simple and
I think could easily be ported to all the platforms we play on. The
module that is wrapped does not need to be internally modified at all. I
included the code for a very simple wrapping below. I also have a small
github project that demonstrates more advanced, full server side
dependency resolution (packaging multiple modules in one response) for
server side JavaScript servers [2].

- Migration path -

I believe any single-version radical backwards-incompatible change in
Dojo is not a good thing for our community. I think an appropriate path
forward would be multi-phase:
1. Incorporate RequireJS module loader into Dojo, but keep existing
support for dojo.require (Dojo 1.6)
2. Build server side module wrappers that support wrapping CommonJS and
Dojo modules with CommonJS transport format (concurrently).
3. Migrate all Dojo code to CommonJS format and incorporate the
transport format wrapping in our build so all module can be loaded async
(Dojo 1.7)
4. Deprecate dojo.require (Dojo 1.8)
5. Remove dojo.require (eliminate all sync module loading) (Dojo 2.0)


[1] http://wiki.commonjs.org/wiki/Modules/Transport/D
[2] http://github.com/kriszyp/transporter
Pseudo code for server side module wrapping:

var content = getFileContents(path);
var depsObject = {};
content.replace(/require\s*\(\s*['"]([^'"]*)['"]\s*\)/g, function(t,
moduleId){
  depsObject[moduleId] = true;
});
write("require.define({" + JSON.stringify(path.substring(0, path.length
- 3)) + ":function(require, exports, module){");
write(contents);
var deps = [];
for(var i in depsObject){
  deps.push(i);
}
write("},[" + deps.join(",") + "])");

Thanks,
Kris
On 12/22/2009 8:55 PM, Alex Russell wrote:
> +1 to this approach.
>
> I'd like to see this as the module system for a Dojo 2.0.
>
> Regards
>
> On Dec 22, 2009, at 6:46 PM, James Burke wrote:
>
>   
>> I have been working on a new module loader, RunJS. The goal was to
>> avoid XHR calls and eval, and have something simpler that worked cross
>> domain and still had multiversion support. And something that did not
>> need a server process to transform modules.
>>
>> I have also tried to keep it in-line with CommonJS as much as possible
>> -- I use "some/module" module identifiers now (instead of
>> "some.module"), and modules are not defined/visible in the global
>> scope (this allows multiversion support).
>>
>> I have been posting a bit to the CommonJS list to see if I could bring
>> what I think works best in the browser with the current CommonJS
>> module format. However that group seems to be fine with XHR-based
>> loaders, at least in development, or requiring a server process that
>> transforms CommonJS modules on the fly to something that can be used
>> in the browser.
>>
>> To me, that means treating the browser as a second class citizen for
>> CommonJS modules. Their format cannot run as it is via script tags. So
>> I am not going to bother with trying to get any closer to their
>> format. Instead I plan to push RunJS as much as I can for
>> browser-based toolkits.
>>
>> However, I value your feedback. If you think that is a bad idea, I
>> would like to know. And/or if you do not think RunJS would be a good
>> module loader some time in Dojo's future, that would be good to know.
>>
>> More background is here:
>> http://groups.google.com/group/commonjs/browse_thread/thread/57a46efdcdfce122
>>
>> Info about the latest RunJS:
>> http://code.google.com/p/runjs/wiki/RunJs
>>
>> RunJS now has plugins for i18n bundles, and text files dependencies
>> (think dojo.cache, but *async* XHR calls, and inlined in the JS file
>> via the RunJS build process). Modules can return anything they want
>> for their module definition function (Object, Function, String,
>> Number, whatever), where CommonJS may only be restricted to plain
>> Objects with properties, not even constructor functions. Although they
>> are still debating that point.
>>
>> James
>> _______________________________________________
>> dojo-contributors mailing list
>> dojo-contributors at mail.dojotoolkit.org
>> http://mail.dojotoolkit.org/mailman/listinfo/dojo-contributors
>>     
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at mail.dojotoolkit.org
> http://mail.dojotoolkit.org/mailman/listinfo/dojo-contributors
>
>   


More information about the dojo-contributors mailing list