[dojo-contributors] Dojo 1.9 Release Planning

Kris Zyp kzyp at dojotoolkit.org
Fri Oct 19 15:30:00 EDT 2012

I'll try to fill in more of details of the Observable issue that Ken 
mentioned and how to "fix" it. First, the problem is at least partly due 
to the fact dojo/store basically just inherited the paging convention of 
dojo/data while introducing a new event system, which introduces issues 
when multiple "paged" result sets have events that may cause objects to 
traverse from one page another, and the Observable store wrapper is 
limited to a view of autonomous pages that make it impossible to 
determine exactly where an object should go when it is at a page boundary.

I'll try to illustrate this, to make it clearer. Suppose we did a query:

store.query({...}, {sort:[{attribute:"rank"}], start:0, count: 5});

and it returned sorted objects with the following values for rank:
1, 2, 3, 4, 5

Now suppose the object with the rank of 3 was modified and given a rank 
of 30. The Observable store only has access to the five objects above, 
so after the change, it would sort them and have a new list:

1, 2, 4, 5, 30.

Now, the question is, does the modified object really belong to this 
result set, or does it actually in fact belong to another paged result 
set (like start:30, count: 5, perhaps)? It is impossible to determine 
this without more information.

I suggested three possible solutions:

1. We could add an additional callback function to observe() that could 
be used by the Observable store wrapper to ask the query owner (the 
widget making the query) for next object in the next page (which 
presumably it would have access to). In the example above, it could ask 
the widget for the first object in the next paged query result. If the 
next object has a rank of 6, it would clearly know that the object no 
longer belonged in the current page.

This approach is certainly unattractive from an API perspective, adding 
a very awkward form of additional complexity to the observable 
interface. However, I have tried this out successfully with dgrid.

2. We could have the Observable store requery the underlying store when 
an object is moved to a page boundary, to see what the new result set 
should be after the change. When an event leaves Observable in ambiguous 
state, it would basically ask the store for the authoritative 
information on what the result sets should now be.

This approach is attractive from an API perspective since we could leave 
the interface alone. However, this has the potential to introduce a 
serious performance regression. One of the purposes of the Observable 
interface is to allow for incrementally update result sets without 
requerying, and if requerying forces additional requests to a server (if 
you are using JsonRest store), this can have a large negative impact. We 
have also built a patch for this change and used it with dgrid as well.

3. We could change the paging mechanism to be a post-query subset call, 
instead of being driven by query options. With this approach, we add a 
method to the query results, to get a page of data out of the results. 
Probably the best choice would be slice(), so the API matches that of 
standard arrays. Then, to get a page of data we would do something like 

results = store.query({...});
firstPage = results.slice(0, 10); // get the first 10 objects

And then to monitor for changed data, we would call observe() on the 
single total results, rather than having to observe() every single page 
for changes, and trying to weave the events through different pages and 
having the widget coordinate these events with each page of data.

Observable would then be operating on the total results, instead of 
individual pages, and therefore would have sufficient information to 
determine the correct index for objects as they change.

I think this is attractive from API and performance perspective, but it 
is more involved and less back-compat change. Stores would need to be 
upgraded in order to able to leverage this new API. And for some stores 
this is more sophisticated than simply adding a new method that acts 
like array slice. To properly implement this with the JsonRest store 
would entail deferring the actual HTTP request from a query() until the 
results were actually accessed or sliced. When you call query(), a 
result set object would need to be returned, and then when you call 
forEach the HTTP request would actually be sent, or if you called 
slice() an HTTP request with the Range header would be sent.

I think from broader perspective of API usage, this would actually be a 
good improvement to dojo/store. Specifically, I think it would be very 
appealing and much cleaner to be able to simply pass a result set object 
to widget if it is responsible for displaying a set of data, and it 
could slice and iterate it as it sees fit (and we could also add a 
function version of sort(), perhaps). Right now, widgets require both 
the store and query to use on it, which is much more information than it 
really needs, if all it needs to do is iterate over objects.

#3 seems like the best long-term solution to me. However, I don't know 
if we want to tackle this in 1.x, we may want to use #1 or #2 for 1.x. 
Or if anyone has a better idea...


On 10/19/2012 12:49 PM, Kenneth G. Franqueiro wrote:
> Getting back to the Dojo 1.9 release plan in general, it occurs to me
> that there may be something in our best interests to look at that hasn't
> been considered yet: "fixing" dojo/store/Observable.
> dojo/store/Observable attempts to provide an API allowing store
> consumers to react directly to individual changes in data without
> necessitating an entire refresh.  However, due to its nature of
> attaching to individual query results, it falls flat in some cases when
> a consumer attempts to maintain multiple pages of a result set (i.e.
> different start/count, same query/queryOptions otherwise).
> People have been starting to run into issues with dgrid due to
> limitations/quirks of Observable itself, and I think this is something
> we seriously need to consider if it is becoming a tripping point for
> widgets and other consumers which need to rely on it heavily.  Colin,
> myself, and Kris had a bit of discussion in the past about potential
> options for going forward with this for 2.0, but if we could prioritize
> it as a Dojo 1.9 item I think it has potential to benefit a lot of
> people, rather than leaving 1.x users hanging.  Naturally I'm a bit
> concerned if it necessitates having a new separate observable API
> alongside the current one, but that may be what's required.
> --Ken
> On 10/16/2012 5:00 AM, Kitson Kelly wrote:
>> On 15 October 2012 15:02, Christophe Jolif <cjolif at gmail.com
>> <mailto:cjolif at gmail.com>> wrote:
>>      If there are questions / remarks I'm happy to discuss that this week
>>      on the mailing list as I won't be at the meeting. Otherwise I do
>>      plan to attend the week after (24) but that might a bit late to
>>      discuss that (deadline for plan approval in Oct 26 in your document).
>> Mostly, my strategy is to just beat the drum to keep us moving forward.
>>   If we don't reach an obvious consensus this week in regards to 1.9,
>> then, we move out against our baseline.  The main thing is to keep
>> moving forward in a transparent fashion.
>> _______________________________________________
>> dojo-contributors mailing list
>> dojo-contributors at mail.dojotoolkit.org
>> http://mail.dojotoolkit.org/mailman/listinfo/dojo-contributors
> _______________________________________________
> dojo-contributors mailing list
> dojo-contributors at mail.dojotoolkit.org
> http://mail.dojotoolkit.org/mailman/listinfo/dojo-contributors

More information about the dojo-contributors mailing list