[dojo-contributors] Thoughts on API documentation

Tom Trenka ttrenka at gmail.com
Sat May 31 13:07:35 EDT 2008


Before I volunteered to help out Neil and design the API site, I had
looked into other documentation options, and briefly discussed them
with Dylan and Adam (separately).  The only other option out there
that came close to what we needed was this:

http://jsdoctoolkit.org/

There seem to be some confluence of interests with this toolkit--in
particular, the build tool they use to generate the documentation
(which ends up usually as a set of static HTML files) happens to be
very similar to what we use for our own tools, which is pretty neat.
However...

...if you dive into it, you'll find (as I did) that it doesn't come
close to what we need.  In particular, the dojo.declare syntax used
throughout Dojo makes it difficult to try to adapt a documentation
tool for it.  You can do custom tagging with it but in the end I think
the effort would be at least as great as the one Neil undertook.

---

I am with James on some of what he would want out of an API; offline
in particular would be a blessing.  However, I'm under the
impression/opinion that we can use the tools Neil has developed--and
continues to develop--to achieve these goals.  IMHO its a matter of
really defining what we need, and then doing it.  A good portion of
what Neil wrote was basically defined by what he felt was the
requirements...I know that Neil did this uber-cool dojox.help thing
that reads off the API site (I think), that might be an option.

I know there is an effort underway at SitePen to help address offline
docs; I don't believe I'm allowed to talk about it at all, but Dylan
might be willing to volunteer some information about it.

As I did with the API site, I'm willing to help out any kind of
offline formatting.  I think it's probable that we can do something
where we generate an offline version with every release, in the same
way we create a default build for every release (at least I seem to
remember Neil mentioning this is possible, and its one of his goals).
I'll talk to Neil about it.

One other thing I should mention (I believe I asked Neil about this at
one point and he confirmed that it's possible): his API tool can read
comments in a JavaDoc-like format as well as the format we've been
using (i.e. starting with /** ).  It might be nice if we were able to
dual-use some of the internal comments in the source, so that a fairly
vanilla JavaDoc representation could also be created.  Neil would have
to speak more to this one though.

regards,
trt

On Sat, May 31, 2008 at 10:26 AM, Nathan Toone
<toonetown at dojotoolkit.org> wrote:
> Just as a quick comment, in the past, we have used JSDoc
> (jsdoc.sourceforge.net) to document our javascript source code.  It uses a
> javadoc-like syntax, and also supports inheritance.  It outputs static HTML
> files - but structured by package.  I'm guessing (I haven't looked too much
> into the details) that it could be fairly easily-extended to recognize the
> provides/requires/declares syntax of Dojo - and it might address all the
> issues that have been raised (support inheritance, generate static "offline"
> files, etc.)
> -Nathan
> On May 31, 2008, at 12:36 AM, Neil Roberts wrote:
>
> I'm not opposed to switching systems, though I hope to touch on a few of the
> limitations as I make notes below. It's true that I haven't had as much time
> as I'd like to dedicate to the effort. In particular, I appear to simply be
> unable to explain any of this, which has probably caused the most problems.
> For those unfamiliar with the history, I worked on documentation around 0.3,
> but was pushed out around 0.4 for pretty much the same reasons listed below,
> and the current doc system is actually a personal project of mine that I
> showed to Alex who thought it might fill the gap left when Owen (the guy who
> took over docs and produced the 0.4 API tool) left the community.
> Responses will be inline.
> On May 30, 2008, at 7:35 PM, James Burke wrote:
>
> These are some questions/issues I have had with the way we do API
> documentation. I was not sure how to raise them before because I did
> not feel like I saw the end goal clearly. Also, many of you, and in
> particular Neil, are much more talented engineers than I am, and I
> felt like I might be asking silly questions. But I cannot seem to
> resolve these questions/issues for myself, so I am laying them out
> now.
>
> Online vs. Offline
> ====================
> I feel like the process of generating and using API documentation
> should be easier. The use case for using a PHP/DB system seems to be
> to enable comments. Generating a set of "offline" or static API
> documents seems to be a second-tier goal.
>
> Not just for comments. First, the relationships created between the various
> files are hard to see without a structured backend. This is better explained
> by noting that things that aren't in the up-facing require chain might
> modify an object. An example is the dojo-ext stuff, take NodeList-fx for
> example. If I'm looking at the dojo.NodeList object, it's nice to know that
> another file adds to the NodeList object, but that requires some sort
> of knowledgeable backend so that I don't have to look through it again.
> Second, Drupal helped us put it in a pretty package. Drupal also gave us a
> lot of leverage when it came to dealing with the objects.
> This backend could possibly be done in another language in an object
> structure and serialized/deserialized at runtime.
> Also, we currently do have the abilities to generate a set of offline
> documents, which I wrote for Aptana, it's just not exposed. I'd be happy for
> anyone to volunteer to create that UI.
>
> I would rather see the primary goal to be generating offline/static
> API documents that could be delivered as part of the distributions,
> then use a JSONP-type API to the Dojo server to pull in comments. That
> way, I could at least view the API while offline or in a faster
> fashion than depending on the availability and responsiveness of
> dojotoolkit.org (which has been an issue, and will be in the future).
>
> We already have a JSONP API, adding comment support should not be a problem.
>
> Going the "use the server just for comments" also feels like less of a
> resource impact on our DB and our website: just limiting it to
> managing comments. It also means if the server cannot respond, I can
> still view docs.
>
> Also, having a way to generate docs without installing and configuring
> PHP would be nice.
>
> As I sort of outlined above, the searchable data structure to reduce all of
> our files into distinct objects is the problem here. I'm sure it can be
> solved without PHP, but it would still need to be solved.
>
> Custom vs. off the shelf
> ==================
> I am not sure we should be developing a custom documentation system
> anyway. We clearly do not have time for it. It took a very long time
> to get it up and running (no one's fault, we're all busy -- I
> certainly could have done more doc comments), and it still feels less
> than 100%. Going with an "off the shelf" documentation solution from
> the start seems like it would have saved time: we could have started
> sooner and gotten quicker feedback on the doc output.
>
> Maybe things like dojo.provide/dojo.require makes using an existing
> doc system hard to do, I do not know. Feel free to enlighten me. But
> maybe the way to solve it is enhancing an existing tool vs making our
> own.
>
> Short answer: inheritance and custom object manipulation. Even the tools
> that give some support for our custom functions
> (require/provide/extend/mixing/declare) simply don't do inheritance. I'd be
> glad to be proven wrong.
>
> Doc format
> ===================
> I am not sure developing our own syntax for the comments has worked
> out. I appreciate the dynamic nature of JavaScript, but it seems like
> we have to go through some hoops to get our syntax to work out anyway.
> It seems like we could just take the hit of jumping through hoops to
> use another syntax. I do not feel it was a net gain.
>
> I am also not a fan of having to learn markdown. It bothers me that I
> have to use a white space-aware syntax, and a syntax I do not use on a
> regular basis. I would prefer to use HTML. HTML is something all
> users/contributors already know if they are doing web development. But
> I appreciate this could be a bikeshed, personal taste issue.
>
> It's sort of a bikeshed issue. I'm a big believer in documentation written
> in source code being meaningful both in a parsed format and as it's seen in
> source. I have never wanted full-blown descriptions to be written in source
> code, but I'm quite aware that pretty much the entire community wants them.
> Markdown seemed like a good compromise between readable and parsable.
>
> Conclusion
> ==================
> I appreciate that we were trying to do something that imposed less of
> a tax on contributors to document the code, and to document JavaScript
> better. However, any sort of tax reduction on the contributor front
> seems marginal, particularly compared to the tax we have imposed on
> our users:
> - having to wait so long for documentation
> - having to depend on the availability of dojotoolkit.org
> - not making it clear what version api.dojotoolkit.org is pointing to
> or how to get specific versions
>
> I have been working on versioning, but I'll be stopping if we're going to go
> in a different direction with documentation.
>
> - a fairly high cost of entry to use the doc system to generate docs
> (installing/configuring PHP, dealing with the output).
> - no offline docs by default.
>
> And for me at least, having to learn a "non-standard" doc format and
> having to use markdown were additional taxes on me -- learning our doc
> system did not result in a skill I can apply to other codebases, since
> it is custom.
>
> What we have is certainly useful, but it does not seem like we
> documented our APIs better than other toolkits.
>
> Possible Options
> =================
> So, again, this is just my opinion. I am fairly ignorant on
> documentation systems, so I could just be seeing personal problems and
> missing the big issues our system solves. Please clue me in.
>
> My biggest issue is the lack of offline/static docs that are portable
> and easy to generate. The off-the-shelf solution and custom doc syntax
> issues are lesser concerns, and perhaps the problem domain requires
> them.
>
> If at least some of these issues seem valid, here are some possible
> ways to address it. These are ordered from perhaps easiest to hardest,
> or at least these things could be done in stages.
>
> - Output static HTML by default. Maybe use the XML intermediate format
> from the doc system now and do some
> transform on that?
>
> - Deliver the static HTML on download.dojotoolkit.org.
>
> - Convert api.dojotoolkit.org to use static HTML, and create a
> versions page. Do not generate/serve docs for trunk on
> api.dojotoolkit.org: trunk users can look at code (or maybe use rhino
> task mentioned below?).
>
> - Create a JS library and JSONP API that loads comments from the
> server. This library should fail gracefully if the host is not
> available.
>
> - Convert doc building to use Rhino so a "docs" command can be run
> from util/buildscripts or util/docscripts. Or something that does not
> require extra tools other than what we can check into the tree or
> something easy to download. Making sure PHP is installed and
> configured, to me, is an onerous requirement.
>
> - Convert code to use off-the-shelf documentation system? I do not
> really care, just something that seems to be in general use that
> supports HTML formatting. I appreciate this might be a harder,
> bikeshed task to consider, and maybe they do not work for things like
> dojo.require/provide. But if not, is enhancing an existing tool an
> option instead of building out our own?
>
> Sorry for the long email. I do not want to take away from the progress
> we have already made, or retread old discussions that I do not recall.
> I just have had a hard time resolving these things for myself, and I
> am hoping to get help with it.
>
> James
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at dojotoolkit.org
> http://turtle.dojotoolkit.org/mailman/listinfo/dojo-contributors
>
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at dojotoolkit.org
> http://turtle.dojotoolkit.org/mailman/listinfo/dojo-contributors
>
>
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at dojotoolkit.org
> http://turtle.dojotoolkit.org/mailman/listinfo/dojo-contributors
>
>



More information about the dojo-contributors mailing list