[caldeveloper-l] sync-tokens vs PROPFIND

Wed Nov 8 05:07:13 PST 2017

Unfortunately, our infrastructure doesn't allow streaming results --- we
need to marshal responses in memory before returning them.

Please note (I don't think it was sufficiently clear in my initial email):
this appears to be happening during the *initial sync* phase. Documents
such as http://sabre.io/dav/building-a-caldav-client/ imply that
sync-operation is only of use once the initial sync is complete and you're
looking for changes. The RFC doesn't say, when way or another.

Essentially, I have two concerns here: firstly, we need to be standards
conforming when faced with huge queries, and secondly, while being
technically correct is the best kind of correct, we also need to return
useful results given the huge number of ancient and buggy clients we have.

Another look at the RFC suggests that a perfectly reasonable thing to do
when faced with a very large query is to return what we can with a 507
number-of-matches-within-limits error. Apart from being more conformant,
this should allow them to sync something, rather than nothing at all, which
is an improvement. (Although it's quite possible that the buggy client will
see the 507, assume it's a fatal error, and ignore the payload.)

If we add the sync token as well, just on the off chance that clients might
honour it, will this cause any problems? The RFC suggests not, but...

On Wed, 8 Nov 2017 at 10:18 CardBook <cardbook.thunderbird at gmail.com> wrote:

> there already is such a document, which seems to have recently changed :
> http://sabre.io/dav/building-a-carddav-client/
>
> Philippe
>
>
> Le 07/11/2017 à 22:09, Michael Douglass a écrit :
>
> We're working on such a thing - take a look at
> https://devguide.calconnect.org/
>
> Still a work in progress...
>
>
> On 11/7/17 15:42, Tim Hare wrote:
>
>
> I'm not (yet) a developer at that level, but seems to me there might be an
> advantage to all calendar software developers if CalConnect published an
> open-source set of 'functional' examples of how to do things "How to
> retrieve the user's events", "How to send a meeting invitation", etc.   It
> might help drive adoption of 'best practices' methods and make it easier on
> server vendors for sure.  Or is this what the API group is hoping to
> accomplish?  In an event, for client developers, it's usually the UX that
> differentiates products, the user rarely sees or thinks about the 'back
> end'  (exactly as in e-mail).
>
> Tim Hare
> Interested Bystander, Non-Inc.
>
> On 11/7/2017 12:00 PM, Ken Murchison wrote:
>
>
> My guess is that if the client is blindly requesting the ETag of every
> resource using a PROPFIND rather than a sync-collection REPORT, that the
> client will have no clue what a sync-token is if you return it
>
> I'd push back on the client developer and educate them that
> sync-collection is almost mandatory to implement for any well-behaved
> CalDAV/CardDAV client.
>
> Is the PROPFIND not working because Google can't handle it or the client
> times out waiting for the response?  If the latter, we at Fastmail recently
> experienced the same thing with requests that require a huge response to be
> generated.  We fixed this by updating the Cyrus IMAP server so that
> responses to PROPFIND and REPORT get "streamed" to client using chunked
> transfer-encoding.  This keeps the client from timing out.
>
>
> On 11/07/2017 11:38 AM, David Given wrote:
>
> Hello everybody,
>
> I have an RFC question I'd appreciate some feedback on...
>
> We have a situation where a Calendar client is trying to get the etags of
> every event in an enormous calendar in a single query. Naturally, this
> isn't working.
>
> They're doing this:
>
> PROPFIND /caldav/v2/$USERNAME/events
> <?xml version="1.0" encoding="UTF-8"?>
> <A:propfind xmlns:A="DAV:">
>   <A:prop>
>     <A:getcontenttype/>
>     <A:getetag/>
>   </A:prop>
> </A:propfind>
>
> As you can see, they're not asking for a sync token.
>
> RFC6578 section 3.6 provides a mechanism where clients can ask for
> paginated responses, where the server returns partial data with a 507
> status code, and a new sync token so they can continue the query. It is a,
> however, a bit ambiguous as to whether this is supported for methods other
> than REPORT. The interwebs suggest it is (but I'd appreciate a
> clarification here).
>
> My question is. however:
>
> Is it permissable to return a sync-token (and expect the client to honour
> it) /even if the client's original query didn't ask for it/?
>
>
> I'm hoping the answer is going to be yes, but if it's no --- any
> suggestions?
>
> Thanks!
>
> David Given
>
> dtrg at google.com <mailto:dtrg at google.com> <dtrg at google.com>
>
>
>
>
> _______________________________________________
> caldeveloper-l mailing list
> caldeveloper-l at lists.calconnect.org
> http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
> --
> Ken Murchison
> Cyrus Development Team
> FastMail Pty Ltd
>
>
> _______________________________________________
> caldeveloper-l mailing list
> caldeveloper-l at lists.calconnect.org
> http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
>
>
>
> _______________________________________________
> caldeveloper-l mailing list
> caldeveloper-l at lists.calconnect.org
> http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
>
>
> We're working on such a thing - take a look at
> https://devguide.calconnect.org/
>
> Still a work in progress...
>
> On 11/7/17 15:42, Tim Hare wrote:
>
> I'm not (yet) a developer at that level, but seems to me there might be an
> advantage to all calendar software developers if CalConnect published an
> open-source set of 'functional' examples of how to do things "How to
> retrieve the user's events", "How to send a meeting invitation", etc.   It
> might help drive adoption of 'best practices' methods and make it easier on
> server vendors for sure.  Or is this what the API group is hoping to
> accomplish?  In an event, for client developers, it's usually the UX that
> differentiates products, the user rarely sees or thinks about the 'back
> end'  (exactly as in e-mail).
>
> Tim Hare
> Interested Bystander, Non-Inc.
> On 11/7/2017 12:00 PM, Ken Murchison wrote:
>
> My guess is that if the client is blindly requesting the ETag of every
> resource using a PROPFIND rather than a sync-collection REPORT, that the
> client will have no clue what a sync-token is if you return it
>
> I'd push back on the client developer and educate them that
> sync-collection is almost mandatory to implement for any well-behaved
> CalDAV/CardDAV client.
>
> Is the PROPFIND not working because Google can't handle it or the client
> times out waiting for the response?  If the latter, we at Fastmail recently
> experienced the same thing with requests that require a huge response to be
> generated.  We fixed this by updating the Cyrus IMAP server so that
> responses to PROPFIND and REPORT get "streamed" to client using chunked
> transfer-encoding.  This keeps the client from timing out.
>
> On 11/07/2017 11:38 AM, David Given wrote:
>
> Hello everybody,
>
> I have an RFC question I'd appreciate some feedback on...
>
> We have a situation where a Calendar client is trying to get the etags of
> every event in an enormous calendar in a single query. Naturally, this
> isn't working.
>
> They're doing this:
>
> PROPFIND /caldav/v2/$USERNAME/events
> <?xml version="1.0" encoding="UTF-8"?>
> <A:propfind xmlns:A="DAV:">
>   <A:prop>
>     <A:getcontenttype/>
>     <A:getetag/>
>   </A:prop>
> </A:propfind>
>
> As you can see, they're not asking for a sync token.
>
> RFC6578 section 3.6 provides a mechanism where clients can ask for
> paginated responses, where the server returns partial data with a 507
> status code, and a new sync token so they can continue the query. It is a,
> however, a bit ambiguous as to whether this is supported for methods other
> than REPORT. The interwebs suggest it is (but I'd appreciate a
> clarification here).
>
> My question is. however:
>
> Is it permissable to return a sync-token (and expect the client to honour
> it) *even if the client's original query didn't ask for it*?
>
> I'm hoping the answer is going to be yes, but if it's no --- any
> suggestions?
>
> Thanks!
>
> David Given
> dtrg at google.com
>
>
>
> _______________________________________________
> caldeveloper-l mailing listcaldeveloper-l at lists.calconnect.orghttp://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
> --
> Ken Murchison
> Cyrus Development Team
> FastMail Pty Ltd
>
>
>
> _______________________________________________
> caldeveloper-l mailing listcaldeveloper-l at lists.calconnect.orghttp://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
>
>
>
> _______________________________________________
> caldeveloper-l mailing listcaldeveloper-l at lists.calconnect.orghttp://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
>
>
> _______________________________________________
> caldeveloper-l mailing listcaldeveloper-l at lists.calconnect.orghttp://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
> _______________________________________________
> caldeveloper-l mailing list
> caldeveloper-l at lists.calconnect.org
> http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.calconnect.org/pipermail/caldeveloper-l-calconnect.org/attachments/20171108/26dd0f22/attachment-0001.html>