[dojo-contributors] Fwd: Using Dojo 1.6 with Closure -- document attached

Stephen Chung Stephen.Chung at intexact.com
Thu Mar 3 23:24:56 EST 2011


Hi Ben,

I have earlier sent my thoughts on AMD to Bill Keese, which I'll reproduce 
below:

I think the AMD format will be a great help in simplifying the source tree 
for processing by Closure.  Actually, in order to make a lot of things work 
within the restrictions of the compiler, the existing code must be modified 
in a way that makes it looks more like an AMD module anyway.  The problem 
areas that currently prevent optimizations by the compiler are all related 
to inter-module connections and ad-hoc dependencies, something that the AMD 
format should get rid of.

Turning Dojo into a vast collection of loosely-coupled modules will 
eliminate needs for “root” namespaces like “dojo”, “dijit” etc. and 
individual, small modules are much easier to optimize, inline, eliminate 
etc.

The RequireJS syntax is actually simpler to process than the normal 
“dojo.provide/require” syntax – that is because “define” automatically 
encapsulates a module while dojo.provide/require is much looser.  As long as 
the code is written to stay within the module (Dojo 1.6 currently isn’t, but 
I suppose 2.0 will be) and not break its boundary, it will be much easier to 
adapt to Closure compilation.

For example, a made-up string module:

    define (“dojo/string”, [ “dojo”, “text/formatters” ], function(dojo, 
fmt) {
        var str = {};
        str.format = function ...
        ...
        return str;    // AMD-Result
    });

can be easily converted, via a simple transformation pass, into the 
following format for Closure:

    var global_dojo_string = {};                 // define converts to this

    goog.scope(function() {
        var str = global_dojo_string;         // var str = {} converts to 
this
        var dojo = global_dojo;                 // AMD parameters convert to 
this
        var fmt = global_text_formatters;

        (function() {
                str.format = ...                          // Everything 
inside the define block without the return
                ...
        })();
    });

The converted code looks ugly, but as long as the code inside the wrapper 
closure does not touch anything outside the module, it is extremely 
efficient to be optimized by the Closure Compiler.  So we can write 
beautiful Dojo code, while having it transformed into the ugly format liked 
by the compiler for optimization.

Based on my experience, I dare say that the AMD format enables the full 
feature set of the Closure Compiler to be used, and so the quicker Dojo 
moves to totally AMD-modularized the better.  I am really only talking about 
AMD "format" here because, obviously, modularized loading is not required 
after a merged build, so it is really a matter of how AMD enables people to 
have the cake (quick loading for modularized systems) and also eat it all 
(fully optimized builds).

I’ve also looked into the intro for bdbuilder and it looks like a fantastic 
piece of work.  The parsing part should enable transforms to be written for 
almost any type of optimization.  However, I doubt whether it would be an 
efficient use of time to duplicate much of what the Closure Compiler is 
already offering today.  My feel is that bdbuilder can be used to add a vast 
variety of features (perhaps additional goodies tied to Dojo syntax) that 
Closure does not address, and use Closure for the standard set of 
optimizations.  Of course, a transform can be written for bdbuilder to 
automatically convert a set of AMD sources into a Closure-friendly format.

- Stephen


-----Original Message----- 
From: ben hockey
Sent: Friday, 04 March, 2011 1:56 AM
To: dojo-contributors at mail.dojotoolkit.org
Subject: Re: [dojo-contributors] Fwd: Using Dojo 1.6 with Closure --  
document attached

you mention AMD a few times and i agree with you that i would expect
that moving to AMD and flattening namespaces should help with
compression.  it will be interesting to see how much of an effect it
has.  i contributed a patch to requirejs[1]  that would allow you to set
options for the closure compiler in your build profile.  i didn't do
much with it because requirejs itself is not written in a way that suits
the advanced optimizations and i didn't have the resources to try and
adapt it for this.  also, after james implemented the build using uglify
and nodejs, i just didn't want to spend the time waiting for rhino anymore.

ben...

[1] https://github.com/jrburke/requirejs/pull/34

On 3/3/2011 3:47 AM, Stephen Chung wrote:
> Hi Rawld,
>
> You've raised very valid questions, and I think you're absolutely right.
> The Closure Compiler is not for everybody.  Its restrictions, pain and
> limitations must be counter-acted by huge benefits; if there are no large
> benefits, then it is really not worth to use it on a project.
>
> Currently, I think that the compiler is worthwhile to use only on projects
> that require very small, very tight JavaScript code (e.g. for use on a
> mobile device).  With more CPU and RAM on mobile devices in each new
> generation, in a year or two even this need may go away.
>
> Regarding some of your points:
>
> 1) Properties renaming: my opinion is that it is really valuable only for
> obfuscation purposes, not actually saving bits, although I did get a 
> roughly
> 1/3 smaller file after running through the compiler than with the regular
> build (compress with closure in simple mode).  I expected the gzipped
> difference to be minimal, but strangely the difference is again roughly 
> 1/3.
> I suspect that is more due to elimination of not-used functions than due 
> to
> variables renaming.
>
> 2) Dead code removal: Dojo is already very modularized so there is much 
> less
> to gain.  Moving to AMD format should make it even better.  However, even
> for highly modularized code, I remember based on Google's analysis (that I
> have read somewhere) that roughly 1/3 to 1/2 of library code is typically
> not used -- not every application calls every function.  For example, it 
> can
> potentially reduce the essential Dojo core portion to, say, to 50K from 
> 89K,
> depending on what functions are used.  That savings of 40K really is not
> much of a benefit, especially when the web app is 500KB and up...
>
> * Note: Due to the particularities of the language, I believe that any
> dead-code removal algorithm for JavaScript must eventually come up with 
> the
> same roadblocks faced by the programmers of the Closure compiler and will
> require similar restrictions, so my point is that in *any* dead-code
> elimination scenario, we'll have to do similar things to make sure that 
> the
> code is easy to analyze.  Therefore, the learnings from using the Closure
> compiler will be beneficial regardless.  Now, it may not be that we'll
> always *need* to remove dead-code...
>
> 3) What I find beneficial is flattening of namespaces and virtualization 
> of
> prototype methods.  These have great performance benefits, especially on
> slower mobile device.  However, with fast mobile CPUs and better browsers,
> or with the move to AMD format (which eliminates namespacing), namespace
> flattening will be superceded.
>
> 4) Virtualization of prototype methods currently cannot be done with
> dojo.declare'd classes.  However, it can be done for third-party libraries
> though.  I refactored a library that I use (LINQ.js) and Closure largely
> eliminated the performance penalties from using LINQ by virtualizing 
> entire
> call chains.
>
> 5) Function inlining -- currently this has great performance benefits when
> it happens (though the restrictions are so severe that it rarely happens).
> It is great for single-trace code that is written in a way that makes it
> flexible to re-configure with different options but automatically inlined
> and have all these abstractions unwrapped by the compiler -- something 
> like
> dependency injection which can produce an optimized build for each config.
> In fact, this is how Google handles i18n -- an optimized build for each
> locale.
>
> 6) Code rewriting -- can probably be done by any build processor with a
> JavaScript parser.  I notice that Uglify already does much of what Closure
> does.
>
>
> So, as much as I like the Closure Compiler, I must admit that you are 
> right
> in many aspects.  Especially that, when we move to the next few 
> generations
> of mobile devices, and Dojo moving to AMD, there may not be a need to 
> remove
> dead code, flatten namespaces, virtualize prototype functions and etc.
> Renaming of properties is only valuable for obfuscation (although for many
> commercial web apps, this is a great benefit).  Inlining only valuable for
> making optimized builds based on a settings file.
>
> So, to wrap up my long rantings, improvements to mobile hardware/browser 
> and
> to Dojo will render the Closure Compiler less useful, except for several
> potential areas:
>
>      - Very small projects that require the smallest downloads (where 
> savings
> a few KB's on dead code elimination on the library actually makes sense)
>      - Using Dojo with non-Dojo third-party library code (that is written 
> in
> a style to take advantange of prototype virtualization)
>      - Obfuscation of application logic
>      - Making optimized builds based on a (large) set of settings that
> permeates throughout the application
>
> - Stephen
>
> -----Original Message-----
> From: Rawld Gill
> Sent: Thursday, 03 March, 2011 3:30 PM
> To: dojo-contributors at mail.dojotoolkit.org
> Cc: Kris Zyp ; Stephen Chung
> Subject: Re: [dojo-contributors] Fwd: Using Dojo 1.6 with Closure --
> document attached
>
> Hi Stephen,
>
> Thanks for sharing this excellent analysis (and Kris for bringing it to 
> the
> list); I know everybody is aware of the amount of work that goes into
> something like this.
>
> I've been doing some research/dev/thinking about optimization for a while
> now.
> Here are a few observations about optmization in general and closure's
> advanced mode in particular...
>
> re the advantages of renaming to short names:
> Since symbol renaming causes lots of problems, it may be wise to ask, "is 
> it
> worth it?" The answer may be surprising. We've already has a discussion on
> this list about the futility of renaming "dojo" to "d". Bill even prooved
> that
> it *made the gzipped code larger*!!
>
> I did an experiment last year where I renamed all the variables in a 
> medium-
> sized project to 2 chars. The different between long/short name versions
> when
> both were  simple-mode-compiled + gzipped was very close to zero (I don't
> even
> remember which was smaller).
>
> My feeling (no hard data to prove) is that lexical renaming (as in simple
> mode) is worth the cost (zero). Complex renaming w/in the dojo lib for
> versions intended for general use (e.g., something on a CDN) is probably 
> not
> worth it; complex naming for a built project *may* be worth the effort.
>
> re elimination of dead code:
> Dead code is just one kind of advanced optimization that can be made if 
> the
> optimizer does real data flow and control flow analysis. (btw, you 
> mentioned
> that closure did not handle aliases correctly...this causes me concern for
> how
> good it really is). Some of these optimizations are localized (e.g., 
> common
> subexpression elimination); some global (e.g., elimination of a function
> that's never called). So...
>
>    * given the quality of dojo, I doubt there'll be a huge gain by 
> applying
> aggressive local optimizations to the dojo/dijit library
>    * aggressive optimizations sometimes introduce obscure bugs
>    * simple dead-code branch removal does not require these kinds of
> optimizations; it's already planned and being used with has.js, the
> RequireJS
> optimizer, bdBuild, and, even ye-olde dojo 1.x build/programa system
>    * global optimizations (like eliminating never-called functions) may be
> doable using safer/simpler methods than advanced closure.
>
> So, the key economics question is which has higher value: building tooling
> so
> that advanced closure does something useful or tooling our own 
> optimization
> processes?
>
> I don't have an answer...only the question:)
>
> Best,
> Rawld
>
>
>
> On Wednesday 02 March 2011 05:38:43 Kris Zyp wrote:
>> Stephen Chung has been researching the possibility of using Closure's
>> advanced mode on Dojo to create highly compressed builds that eliminate
>> dead code, and shorten property names. Attached is a document detailing
>> his results. This could be very helpful in guiding our decisions about
>> how to achieve extremely compact Dojo applications.
>>
>> Thanks,
>> Kris
>>
>> -------- Original Message --------
>> Subject: Using Dojo 1.6 with Closure -- document attached
>> Date: Tue, 1 Mar 2011 13:06:57 +0800
>> From: Stephen Chung<Stephen.Chung at intexact.com>
>> Organization: Intexact Technologies
>> To: Kris Zyp<kzyp at dojotoolkit.org>
>>
>>
>>
>> Hi Kris,
>>
>> I have put together a TODO document (attached) detailing how to use Dojo
>> 1.6 with the Closure Compiler.
>>
>> Please help take a look, and perhaps put in on somewhere that people who
>> wants to know how to do this can get some references.
>>
>> Thanks!
>>
>> - Stephen
>
>
>
>
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 9.0.872 / Virus Database: 271.1.1/3478 - Release Date: 03/03/11
> 03:34:00
>
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at mail.dojotoolkit.org
> http://mail.dojotoolkit.org/mailman/listinfo/dojo-contributors
_______________________________________________
dojo-contributors mailing list
dojo-contributors at mail.dojotoolkit.org
http://mail.dojotoolkit.org/mailman/listinfo/dojo-contributors






No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 9.0.872 / Virus Database: 271.1.1/3480 - Release Date: 03/04/11 
03:34:00 



More information about the dojo-contributors mailing list