[dojo-contributors] Thoughts on API documentation

Tom Trenka ttrenka at gmail.com
Sat May 31 13:07:35 EDT 2008

Before I volunteered to help out Neil and design the API site, I had
looked into other documentation options, and briefly discussed them
with Dylan and Adam (separately).  The only other option out there
that came close to what we needed was this:


There seem to be some confluence of interests with this toolkit--in
particular, the build tool they use to generate the documentation
(which ends up usually as a set of static HTML files) happens to be
very similar to what we use for our own tools, which is pretty neat.

...if you dive into it, you'll find (as I did) that it doesn't come
close to what we need.  In particular, the dojo.declare syntax used
throughout Dojo makes it difficult to try to adapt a documentation
tool for it.  You can do custom tagging with it but in the end I think
the effort would be at least as great as the one Neil undertook.


I am with James on some of what he would want out of an API; offline
in particular would be a blessing.  However, I'm under the
impression/opinion that we can use the tools Neil has developed--and
continues to develop--to achieve these goals.  IMHO its a matter of
really defining what we need, and then doing it.  A good portion of
what Neil wrote was basically defined by what he felt was the
requirements...I know that Neil did this uber-cool dojox.help thing
that reads off the API site (I think), that might be an option.

I know there is an effort underway at SitePen to help address offline
docs; I don't believe I'm allowed to talk about it at all, but Dylan
might be willing to volunteer some information about it.

As I did with the API site, I'm willing to help out any kind of
offline formatting.  I think it's probable that we can do something
where we generate an offline version with every release, in the same
way we create a default build for every release (at least I seem to
remember Neil mentioning this is possible, and its one of his goals).
I'll talk to Neil about it.

One other thing I should mention (I believe I asked Neil about this at
one point and he confirmed that it's possible): his API tool can read
comments in a JavaDoc-like format as well as the format we've been
using (i.e. starting with /** ).  It might be nice if we were able to
dual-use some of the internal comments in the source, so that a fairly
vanilla JavaDoc representation could also be created.  Neil would have
to speak more to this one though.


On Sat, May 31, 2008 at 10:26 AM, Nathan Toone
<toonetown at dojotoolkit.org> wrote:
> Just as a quick comment, in the past, we have used JSDoc
> (jsdoc.sourceforge.net) to document our javascript source code.  It uses a
> javadoc-like syntax, and also supports inheritance.  It outputs static HTML
> files - but structured by package.  I'm guessing (I haven't looked too much
> into the details) that it could be fairly easily-extended to recognize the
> provides/requires/declares syntax of Dojo - and it might address all the
> issues that have been raised (support inheritance, generate static "offline"
> files, etc.)
> -Nathan
> On May 31, 2008, at 12:36 AM, Neil Roberts wrote:
> I'm not opposed to switching systems, though I hope to touch on a few of the
> limitations as I make notes below. It's true that I haven't had as much time
> as I'd like to dedicate to the effort. In particular, I appear to simply be
> unable to explain any of this, which has probably caused the most problems.
> For those unfamiliar with the history, I worked on documentation around 0.3,
> but was pushed out around 0.4 for pretty much the same reasons listed below,
> and the current doc system is actually a personal project of mine that I
> showed to Alex who thought it might fill the gap left when Owen (the guy who
> took over docs and produced the 0.4 API tool) left the community.
> Responses will be inline.
> On May 30, 2008, at 7:35 PM, James Burke wrote:
> These are some questions/issues I have had with the way we do API
> documentation. I was not sure how to raise them before because I did
> not feel like I saw the end goal clearly. Also, many of you, and in
> particular Neil, are much more talented engineers than I am, and I
> felt like I might be asking silly questions. But I cannot seem to
> resolve these questions/issues for myself, so I am laying them out
> now.
> Online vs. Offline
> ====================
> I feel like the process of generating and using API documentation
> should be easier. The use case for using a PHP/DB system seems to be
> to enable comments. Generating a set of "offline" or static API
> documents seems to be a second-tier goal.
> Not just for comments. First, the relationships created between the various
> files are hard to see without a structured backend. This is better explained
> by noting that things that aren't in the up-facing require chain might
> modify an object. An example is the dojo-ext stuff, take NodeList-fx for
> example. If I'm looking at the dojo.NodeList object, it's nice to know that
> another file adds to the NodeList object, but that requires some sort
> of knowledgeable backend so that I don't have to look through it again.
> Second, Drupal helped us put it in a pretty package. Drupal also gave us a
> lot of leverage when it came to dealing with the objects.
> This backend could possibly be done in another language in an object
> structure and serialized/deserialized at runtime.
> Also, we currently do have the abilities to generate a set of offline
> documents, which I wrote for Aptana, it's just not exposed. I'd be happy for
> anyone to volunteer to create that UI.
> I would rather see the primary goal to be generating offline/static
> API documents that could be delivered as part of the distributions,
> then use a JSONP-type API to the Dojo server to pull in comments. That
> way, I could at least view the API while offline or in a faster
> fashion than depending on the availability and responsiveness of
> dojotoolkit.org (which has been an issue, and will be in the future).
> We already have a JSONP API, adding comment support should not be a problem.
> Going the "use the server just for comments" also feels like less of a
> resource impact on our DB and our website: just limiting it to
> managing comments. It also means if the server cannot respond, I can
> still view docs.
> Also, having a way to generate docs without installing and configuring
> PHP would be nice.
> As I sort of outlined above, the searchable data structure to reduce all of
> our files into distinct objects is the problem here. I'm sure it can be
> solved without PHP, but it would still need to be solved.
> Custom vs. off the shelf
> ==================
> I am not sure we should be developing a custom documentation system
> anyway. We clearly do not have time for it. It took a very long time
> to get it up and running (no one's fault, we're all busy -- I
> certainly could have done more doc comments), and it still feels less
> than 100%. Going with an "off the shelf" documentation solution from
> the start seems like it would have saved time: we could have started
> sooner and gotten quicker feedback on the doc output.
> Maybe things like dojo.provide/dojo.require makes using an existing
> doc system hard to do, I do not know. Feel free to enlighten me. But
> maybe the way to solve it is enhancing an existing tool vs making our
> own.
> Short answer: inheritance and custom object manipulation. Even the tools
> that give some support for our custom functions
> (require/provide/extend/mixing/declare) simply don't do inheritance. I'd be
> glad to be proven wrong.
> Doc format
> ===================
> I am not sure developing our own syntax for the comments has worked
> out. I appreciate the dynamic nature of JavaScript, but it seems like
> we have to go through some hoops to get our syntax to work out anyway.
> It seems like we could just take the hit of jumping through hoops to
> use another syntax. I do not feel it was a net gain.
> I am also not a fan of having to learn markdown. It bothers me that I
> have to use a white space-aware syntax, and a syntax I do not use on a
> regular basis. I would prefer to use HTML. HTML is something all
> users/contributors already know if they are doing web development. But
> I appreciate this could be a bikeshed, personal taste issue.
> It's sort of a bikeshed issue. I'm a big believer in documentation written
> in source code being meaningful both in a parsed format and as it's seen in
> source. I have never wanted full-blown descriptions to be written in source
> code, but I'm quite aware that pretty much the entire community wants them.
> Markdown seemed like a good compromise between readable and parsable.
> Conclusion
> ==================
> I appreciate that we were trying to do something that imposed less of
> a tax on contributors to document the code, and to document JavaScript
> better. However, any sort of tax reduction on the contributor front
> seems marginal, particularly compared to the tax we have imposed on
> our users:
> - having to wait so long for documentation
> - having to depend on the availability of dojotoolkit.org
> - not making it clear what version api.dojotoolkit.org is pointing to
> or how to get specific versions
> I have been working on versioning, but I'll be stopping if we're going to go
> in a different direction with documentation.
> - a fairly high cost of entry to use the doc system to generate docs
> (installing/configuring PHP, dealing with the output).
> - no offline docs by default.
> And for me at least, having to learn a "non-standard" doc format and
> having to use markdown were additional taxes on me -- learning our doc
> system did not result in a skill I can apply to other codebases, since
> it is custom.
> What we have is certainly useful, but it does not seem like we
> documented our APIs better than other toolkits.
> Possible Options
> =================
> So, again, this is just my opinion. I am fairly ignorant on
> documentation systems, so I could just be seeing personal problems and
> missing the big issues our system solves. Please clue me in.
> My biggest issue is the lack of offline/static docs that are portable
> and easy to generate. The off-the-shelf solution and custom doc syntax
> issues are lesser concerns, and perhaps the problem domain requires
> them.
> If at least some of these issues seem valid, here are some possible
> ways to address it. These are ordered from perhaps easiest to hardest,
> or at least these things could be done in stages.
> - Output static HTML by default. Maybe use the XML intermediate format
> from the doc system now and do some
> transform on that?
> - Deliver the static HTML on download.dojotoolkit.org.
> - Convert api.dojotoolkit.org to use static HTML, and create a
> versions page. Do not generate/serve docs for trunk on
> api.dojotoolkit.org: trunk users can look at code (or maybe use rhino
> task mentioned below?).
> - Create a JS library and JSONP API that loads comments from the
> server. This library should fail gracefully if the host is not
> available.
> - Convert doc building to use Rhino so a "docs" command can be run
> from util/buildscripts or util/docscripts. Or something that does not
> require extra tools other than what we can check into the tree or
> something easy to download. Making sure PHP is installed and
> configured, to me, is an onerous requirement.
> - Convert code to use off-the-shelf documentation system? I do not
> really care, just something that seems to be in general use that
> supports HTML formatting. I appreciate this might be a harder,
> bikeshed task to consider, and maybe they do not work for things like
> dojo.require/provide. But if not, is enhancing an existing tool an
> option instead of building out our own?
> Sorry for the long email. I do not want to take away from the progress
> we have already made, or retread old discussions that I do not recall.
> I just have had a hard time resolving these things for myself, and I
> am hoping to get help with it.
> James
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at dojotoolkit.org
> http://turtle.dojotoolkit.org/mailman/listinfo/dojo-contributors
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at dojotoolkit.org
> http://turtle.dojotoolkit.org/mailman/listinfo/dojo-contributors
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at dojotoolkit.org
> http://turtle.dojotoolkit.org/mailman/listinfo/dojo-contributors

More information about the dojo-contributors mailing list