<div dir="ltr"><div>Unfortunately, our infrastructure doesn't allow streaming results --- we need to marshal responses in memory before returning them.</div><div><br></div><div>Please note (I don't think it was sufficiently clear in my initial email): this appears to be happening during the <b>initial sync</b> phase. Documents such as <a href="http://sabre.io/dav/building-a-caldav-client/">http://sabre.io/dav/building-a-caldav-client/</a> imply that sync-operation is only of use once the initial sync is complete and you're looking for changes. The RFC doesn't say, when way or another.</div><div><br></div><div>Essentially, I have two concerns here: firstly, we need to be standards conforming when faced with huge queries, and secondly, while being technically correct is the best kind of correct, we also need to return useful results given the huge number of ancient and buggy clients we have.<br></div><div><br></div><div>Another look at the RFC suggests that a perfectly reasonable thing to do when faced with a very large query is to return what we can with a 507 number-of-matches-within-limits error. Apart from being more conformant, this should allow them to sync something, rather than nothing at all, which is an improvement. (Although it's quite possible that the buggy client will see the 507, assume it's a fatal error, and ignore the payload.)</div><div><br></div><div>If we add the sync token as well, just on the off chance that clients might honour it, will this cause any problems? The RFC suggests not, but...</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, 8 Nov 2017 at 10:18 CardBook <<a href="mailto:cardbook.thunderbird@gmail.com">cardbook.thunderbird@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>there already is such a document, which seems to have recently
changed : <a class="m_9135363959912139503moz-txt-link-freetext" href="http://sabre.io/dav/building-a-carddav-client/" target="_blank">http://sabre.io/dav/building-a-carddav-client/</a></p>
<p>Philippe<br>
</p></div><div text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<div class="m_9135363959912139503moz-cite-prefix">Le 07/11/2017 à 22:09, Michael Douglass
a écrit :<br>
</div>
</div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite">We're
working on such a thing - take a look at
<a class="m_9135363959912139503moz-txt-link-freetext" href="https://devguide.calconnect.org/" target="_blank">https://devguide.calconnect.org/</a>
<br>
<br>
Still a work in progress...
<br>
<br>
<br>
On 11/7/17 15:42, Tim Hare wrote:
<br>
</blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite">
<br>
I'm not (yet) a developer at that level, but seems to me there
might be an advantage to all calendar software developers if
CalConnect published an open-source set of 'functional' examples
of how to do things "How to retrieve the user's events", "How to
send a meeting invitation", etc. It might help drive adoption
of 'best practices' methods and make it easier on server vendors
for sure. Or is this what the API group is hoping to
accomplish? In an event, for client developers, it's usually
the UX that differentiates products, the user rarely sees or
thinks about the 'back end' (exactly as in e-mail).
<br>
<br>
Tim Hare
<br>
Interested Bystander, Non-Inc.
<br>
<br>
On 11/7/2017 12:00 PM, Ken Murchison wrote:
<br>
</blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">
<br>
My guess is that if the client is blindly requesting the ETag
of every resource using a PROPFIND rather than a
sync-collection REPORT, that the client will have no clue what
a sync-token is if you return it
<br>
<br>
I'd push back on the client developer and educate them that
sync-collection is almost mandatory to implement for any
well-behaved CalDAV/CardDAV client.
<br>
<br>
Is the PROPFIND not working because Google can't handle it or
the client times out waiting for the response? If the latter,
we at Fastmail recently experienced the same thing with
requests that require a huge response to be generated. We
fixed this by updating the Cyrus IMAP server so that responses
to PROPFIND and REPORT get "streamed" to client using chunked
transfer-encoding. This keeps the client from timing out.
<br>
<br>
<br>
On 11/07/2017 11:38 AM, David Given wrote:
<br>
</blockquote></blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Hello everybody,
<br>
<br>
I have an RFC question I'd appreciate some feedback on...
<br>
<br>
We have a situation where a Calendar client is trying to get
the etags of every event in an enormous calendar in a single
query. Naturally, this isn't working.
<br>
<br>
They're doing this:
<br>
<br>
PROPFIND /caldav/v2/$USERNAME/events
<br>
<?xml version="1.0" encoding="UTF-8"?>
<br>
<A:propfind xmlns:A="DAV:">
<br>
<A:prop>
<br>
<A:getcontenttype/>
<br>
<A:getetag/>
<br>
</A:prop>
<br>
</A:propfind>
<br>
<br>
As you can see, they're not asking for a sync token.
<br>
<br>
RFC6578 section 3.6 provides a mechanism where clients can
ask for paginated responses, where the server returns
partial data with a 507 status code, and a new sync token so
they can continue the query. It is a, however, a bit
ambiguous as to whether this is supported for methods other
than REPORT. The interwebs suggest it is (but I'd appreciate
a clarification here).
<br>
<br>
My question is. however:
<br>
<br></blockquote></blockquote></blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">
Is it permissable to return a sync-token (and expect the
client to honour it) /even if the client's original query
didn't ask for it/?
<br></blockquote></blockquote></blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">
<br>
I'm hoping the answer is going to be yes, but if it's no ---
any suggestions?
<br>
<br>
Thanks!
<br>
<br>
David Given
<br>
</blockquote></blockquote></blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:dtrg@google.com" target="_blank">dtrg@google.com</a> <a class="m_9135363959912139503moz-txt-link-rfc2396E" href="mailto:dtrg@google.com" target="_blank"><mailto:dtrg@google.com></a>
<br></blockquote></blockquote></blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">
<br>
<br>
<br>
_______________________________________________
<br>
caldeveloper-l mailing list
<br>
<a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a>
<br>
<a class="m_9135363959912139503moz-txt-link-freetext" href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a>
<br>
</blockquote></blockquote></blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">
-- <br>
Ken Murchison
<br>
Cyrus Development Team
<br>
FastMail Pty Ltd
<br>
<br>
<br>
_______________________________________________
<br>
caldeveloper-l mailing list
<br>
<a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a>
<br>
<a class="m_9135363959912139503moz-txt-link-freetext" href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a>
<br>
</blockquote></blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite"><blockquote type="cite">
<br>
<br>
<br>
_______________________________________________
<br>
caldeveloper-l mailing list
<br>
<a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a>
<br>
<a class="m_9135363959912139503moz-txt-link-freetext" href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a>
<br>
</blockquote></blockquote></div><div text="#000000" bgcolor="#FFFFFF"><blockquote type="cite">
<br>
<br>
<fieldset class="m_9135363959912139503mimeAttachmentHeader"></fieldset>
<p>We're working on such a thing - take a look at <a class="m_9135363959912139503moz-txt-link-freetext" href="https://devguide.calconnect.org/" target="_blank">https://devguide.calconnect.org/</a></p>
<p>Still a work in progress...<br>
</p>
<br>
<div class="m_9135363959912139503moz-cite-prefix">On 11/7/17 15:42, Tim Hare wrote:<br>
</div>
<blockquote type="cite">
<p>I'm not (yet) a developer at that level, but seems to me
there might be an advantage to all calendar software
developers if CalConnect published an open-source set of
'functional' examples of how to do things "How to retrieve the
user's events", "How to send a meeting invitation", etc. It
might help drive adoption of 'best practices' methods and make
it easier on server vendors for sure. Or is this what the API
group is hoping to accomplish? In an event, for client
developers, it's usually the UX that differentiates products,
the user rarely sees or thinks about the 'back end' (exactly
as in e-mail).<br>
<br>
Tim Hare<br>
Interested Bystander, Non-Inc.<br>
</p>
<div class="m_9135363959912139503moz-cite-prefix">On 11/7/2017 12:00 PM, Ken
Murchison wrote:<br>
</div>
<blockquote type="cite">
<p>My guess is that if the client is blindly requesting the
ETag of every resource using a PROPFIND rather than a
sync-collection REPORT, that the client will have no clue
what a sync-token is if you return it</p>
<p>I'd push back on the client developer and educate them that
sync-collection is almost mandatory to implement for any
well-behaved CalDAV/CardDAV client.</p>
<p>Is the PROPFIND not working because Google can't handle it
or the client times out waiting for the response? If the
latter, we at Fastmail recently experienced the same thing
with requests that require a huge response to be generated.
We fixed this by updating the Cyrus IMAP server so that
responses to PROPFIND and REPORT get "streamed" to client
using chunked transfer-encoding. This keeps the client from
timing out.<br>
</p>
<br>
<div class="m_9135363959912139503moz-cite-prefix">On 11/07/2017 11:38 AM, David
Given wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hello everybody,<br>
<div><br>
</div>
<div>I have an RFC question I'd appreciate some feedback
on...</div>
<div><br>
</div>
<div>We have a situation where a Calendar client is trying
to get the etags of every event in an enormous calendar
in a single query. Naturally, this isn't working.</div>
<div><br>
</div>
<div>They're doing this:</div>
<div><br>
</div>
<div>PROPFIND /caldav/v2/$USERNAME/events</div>
<div>
<div><?xml version="1.0" encoding="UTF-8"?></div>
<div><A:propfind xmlns:A="DAV:"></div>
<div> <A:prop></div>
<div> <A:getcontenttype/></div>
<div> <A:getetag/></div>
<div> </A:prop></div>
<div></A:propfind></div>
</div>
<div><br>
</div>
<div>As you can see, they're not asking for a sync token.</div>
<div><br>
</div>
<div>RFC6578 section 3.6 provides a mechanism where
clients can ask for paginated responses, where the
server returns partial data with a 507 status code, and
a new sync token so they can continue the query. It is
a, however, a bit ambiguous as to whether this is
supported for methods other than REPORT. The interwebs
suggest it is (but I'd appreciate a clarification here).</div>
<div><br>
</div>
<div>My question is. however:</div>
<div><br>
</div>
<div>Is it permissable to return a sync-token (and expect
the client to honour it) <i>even if the client's
original query didn't ask for it</i>?</div>
<div><br>
</div>
<div>I'm hoping the answer is going to be yes, but if it's
no --- any suggestions?</div>
<div><br>
</div>
<div>Thanks!</div>
<div><br>
</div>
<div>David Given</div>
<div><a href="mailto:dtrg@google.com" target="_blank">dtrg@google.com</a></div>
<div><br>
</div>
</div>
<br>
<fieldset class="m_9135363959912139503mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
caldeveloper-l mailing list
<a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a>
<a class="m_9135363959912139503moz-txt-link-freetext" href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a>
</pre>
</blockquote>
<pre class="m_9135363959912139503moz-signature" cols="72">--
Ken Murchison
Cyrus Development Team
FastMail Pty Ltd</pre>
<br>
<fieldset class="m_9135363959912139503mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
caldeveloper-l mailing list
<a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a>
<a class="m_9135363959912139503moz-txt-link-freetext" href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="m_9135363959912139503mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
caldeveloper-l mailing list
<a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a>
<a class="m_9135363959912139503moz-txt-link-freetext" href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="m_9135363959912139503mimeAttachmentHeader"></fieldset>
<pre>_______________________________________________
caldeveloper-l mailing list
<a class="m_9135363959912139503moz-txt-link-abbreviated" href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a>
<a class="m_9135363959912139503moz-txt-link-freetext" href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a>
</pre>
</blockquote></div>
_______________________________________________<br>
caldeveloper-l mailing list<br>
<a href="mailto:caldeveloper-l@lists.calconnect.org" target="_blank">caldeveloper-l@lists.calconnect.org</a><br>
<a href="http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org" rel="noreferrer" target="_blank">http://lists.calconnect.org/listinfo.cgi/caldeveloper-l-calconnect.org</a><br>
</blockquote></div>