* dealing with expunging of messages.

So my first question relates to whether I'm even w= asting my time.=C2=A0 The imapwiki shows very few servers support it, but a= re any clients known to support it?

Regards

Adrien

--------=_MB38B368CF-6111-4487-B9F4-7BC34898F36B-- From nobody Wed Mar 10 02:07:34 2021 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3B74A3A20F5 for ; Wed, 10 Mar 2021 02:07:32 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.099 X-Spam-Level: X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=isode.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GbqvypebSzFY for ; Wed, 10 Mar 2021 02:07:30 -0800 (PST) Received: from waldorf.isode.com (waldorf.isode.com [62.232.206.188]) by ietfa.amsl.com (Postfix) with ESMTP id A70C83A20F3 for ; Wed, 10 Mar 2021 02:07:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1615370846; d=isode.com; s=june2016; i=@isode.com; bh=/D52Bt78tmzGpxaHhFT0ezFi8z+Nl63L6iD5PFiWTyk=; h=From:Sender:Reply-To:Subject:Date:Message-ID:To:Cc:MIME-Version: In-Reply-To:References:Content-Type:Content-Transfer-Encoding: Content-ID:Content-Description; b=NgBaFyi7gdEA2shNlkdQpIE13NC4l0ZzH/M4c+Ws49K8bvwHIst5u1jLAhdVeJ4+tOgcP5 tqsPj5hsS++hYJyKZuHSmg1lv07T9/pf7GLPUIwbfWUYB91m7hWF/tlY8Lpa8w7pzkbOSh fp+NobDf8dkE4f/Xt4+4aYQVzoHcTQQ=; Received: from [192.168.0.5] (4e697ac1.skybroadband.com [78.105.122.193]) by waldorf.isode.com (submission channel) via TCP with ESMTPSA id ; Wed, 10 Mar 2021 10:07:26 +0000 From: Alexey Melnikov To: Adrien de Croy References:

<40f8d9ec-2387-4e38-819d-fdb4407c5709@spark> <3089d87b-5884-44bf-8eb9-ef5d34f16756@spark> Cc: "imapext@ietf.org" Message-ID: <23240094-4827-f85e-6ad5-6b22bc09905b@isode.com> Date: Wed, 10 Mar 2021 10:07:25 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="------------0A813B625497C00D280239E2" Content-Language: en-GB Archived-At: Subject: Re: [imapext] SEARCHRES RFC5182 X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2021 10:07:32 -0000 --------------0A813B625497C00D280239E2 Content-Type: text/plain; charset=utf-8; format=flowed Content-transfer-encoding: quoted-printable Hi Adrien, On 09/03/2021 23:21, Adrien de Croy wrote: > Hi Everyone > > I've been looking into this extension, but it has a number of problems=20 > and uncertainties in the spec especially: > > * in relation to storing results of ESEARCH extensions, it refers to=20 > them as messages). Can you elaborate/quote specific text which is confusing? > * in relation to reporting errors if the saved results are=20 > incompatible (e.g. stored SEQ but asked to use as UID or vice versa) The document is quite clear that $ references message numbers or UIDs as=20 appropriate in the corresponding context. So a saved result is never=20 incompatible. You can see this in the last paragraph of section 2.1: =C2=A0=C2=A0 Implementation note: server implementors should note that "$" = can =C2=A0=C2=A0 reference IMAP message sequences or UID sequences, depending o= n the =C2=A0=C2=A0 context where it is used.=C2=A0 For example, the "$" marker ca= n be set as =C2=A0=C2=A0 a result of a SEARCH (SAVE) command and used as a parameter to= a UID =C2=A0=C2=A0 FETCH command (which accepts a UID sequence, not a message seq= uence), =C2=A0=C2=A0 or the "$" marker can be set as a result of a UID SEARCH (SAVE= ) =C2=A0=C2=A0 command and used as a parameter to a FETCH command (which acce= pts a =C2=A0=C2=A0 message sequence, not a UID sequence). > * dealing with expunging of messages. I think this is also clear, from Section 2.1: =C2=A0=C2=A0 When a message listed in the search result variable is EXPUNGE= d, it =C2=A0=C2=A0 is automatically removed from the list.=C2=A0 Implementors are= reminded =C2=A0=C2=A0 that if the server stores the list as a list of message number= s, it =C2=A0=C2=A0 MUST automatically adjust them when notifying the client about =C2=A0=C2=A0 expunged messages, as described in Section 7.4.1 of [IMAP4]. If it helps, the best way of implementing this (IMHO) is to store a list=20 of UIDs in some form and translate them to sequence numbers in the=20 server, when needed. When messages are expunged, this list is purged of=20 expunged messages. > So my first question relates to whether I'm even wasting my time.=C2=A0 Th= e=20 > imapwiki shows very few servers support it, but are any clients known=20 > to support it? I don't know the answer to this. Best Regards, Alexey --------------0A813B625497C00D280239E2 Content-Type: text/html; charset=utf-8 Content-transfer-encoding: quoted-printable

Hi Adrien,

On 09/03/2021 23:21, Adrien de Croy wrote:

Hi Alexey

------ = Original Message ------

From: "Alexey Melnikov" <alexey.melnikov@isode.com>

To: "Adrien de Croy" <adrien@qbi= k.com>

Cc: "imapext@ietf.org" <imapext@ietf.org>

Sent: 10/03/2021 11:07:25 pm

Subject: Re: [imapext] SEARCHRES RFC5182

Hi Adrien,

On 09/03/2021 23:21, Adrien de Croy wrote:

=20 =20 Hi Everyone

I've been looking into this extension, but it has a number of problems and uncertainties in the spec=C2=A0especially:

* in relation to storing results of ESEARCH extensions, it refers to them as messages).

Can you elaborate/quote specific text which is confusing?

When the SAVE result option is co= mbined with the MIN or MAX [ESEARCH]
=C2=A0 =C2=A0result option, and n= one of the other ESEARCH result options are
=C2=A0 =C2=A0present, the= corresponding MIN/MAX is returned (if the search result
=C2=A0 =C2=A0i= s not empty), but the "$" marker would contain a single message as
=C2= =A0 =C2=A0returned in the MIN/MAX return item.

It's not clear if $ is storing the result of the MIN, or somet= hing else.=C2=A0 But I guess those are the same for MIN and MAX - the IDs.<= /div>

= $ only stores message IDs, not messages.=C2=A0 Maybe replace "contain a sin= gle message" with "refers to a single message" or "contains the message ID"=

I guess I should have read it a few more times.=C2=A0 Maybe it's not so un= clear.

* in relation to reporting errors if the saved results are incompatible (e.g. stored SEQ but asked to use as UID or vice versa)

The document is quite clear that $ references message numbers or UIDs as appropriate in the corresponding context. So a saved result is never incompatible.

So we save both UIDs and SEQ = and adjust these on expunge, and just use whichever one is appropriate in= the subsequent commands.

Thanks!

Adrien

<= p>

You can see this in the last paragraph of section 2.1:

=C2=A0=C2=A0 Implementation note: server implementors should note th= at "$" can
=C2=A0=C2=A0 reference IMAP message sequences or UID sequences, depen= ding on the
=C2=A0=C2=A0 context where it is used.=C2=A0 For example, the "$" mar= ker can be set as
=C2=A0=C2=A0 a result of a SEARCH (SAVE) command and used as a parame= ter to a UID
=C2=A0=C2=A0 FETCH command (which accepts a UID sequence, not a messa= ge sequence),
=C2=A0=C2=A0 or the "$" marker can be set as a result of a UID SEARCH = (SAVE)
=C2=A0=C2=A0 command and used as a parameter to a FETCH command (whic= h accepts a
=C2=A0=C2=A0 message sequence, not a UID sequence).

* dealing with expunging of messages.

I think this is also clear, from Section 2.1:

=C2=A0=C2=A0 When a message listed in the search result variable is EXPUNGEd, it
=C2=A0=C2=A0 is automatically removed from the list.=C2=A0 Implemento= rs are reminded
=C2=A0=C2=A0 that if the server stores the list as a list of message numbers, it
=C2=A0=C2=A0 MUST automatically adjust them when notifying the client = about
=C2=A0=C2=A0 expunged messages, as described in Section 7.4.1 of [IMA= P4].

If it helps, the best way of implementing this (IMHO) is to store a list of UIDs in some form and translate them to sequence numbers in the server, when needed. When messages are expunged, this list is purged of expunged messages.

So my first question relates to whether I'm even wasting my time.=C2=A0 The imapwiki shows very few servers support it, but are any clients known to support it?

I don't know the answer to this.

Best Regards,

Alexey

--------=_MB7B96A514-F5CC-47C2-937A-7670668BDB8E-- From nobody Thu Mar 11 03:49:00 2021 Return-Path: X-Original-To: imapext@ietfa.amsl.com Delivered-To: imapext@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 617163A1A13 for ; Thu, 11 Mar 2021 03:48:59 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cridland.net Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3RvrRUdNfoqP for ; Thu, 11 Mar 2021 03:48:56 -0800 (PST) Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 68E0A3A1A12 for ; Thu, 11 Mar 2021 03:48:56 -0800 (PST) Received: by mail-wm1-x32d.google.com with SMTP id u5-20020a7bcb050000b029010e9316b9d5so9659462wmj.2 for ; Thu, 11 Mar 2021 03:48:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cridland.net; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=T5hIKFH91ifTD1i3sppRut4Z0Yx2wFcZOVZ44ZlNr4w=; b=e3PGnwxmbGiM8mSBQB/8/XIVxd35Q40iIbvFnA+xkocEyxD7PA0BOkOVmFYAmWRrrl JhEdNixbWOocIyzaPUuDUGwh/u2Mmdb9IiiMXquHL0Rw2QwV36I9MhyaQTtEEBCkcVVF b1FOBmZ0waDFJg6mEKxF/0/3QQy4OZt8U4084= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=T5hIKFH91ifTD1i3sppRut4Z0Yx2wFcZOVZ44ZlNr4w=; b=N4yibmS1iPnqFm3BGOiPcPoamV9Dr0zt/PJ8rk3VXD14drsPU9p6x9kHFg7hDUpiyC vWwbwuvfNvWzm1Q/Pfyj3Awi1QIGlXvn2Q+5LCGs725RarFdnCttcV78E+xscfMXnWka YxO4v6PoaqDBWzIW0KnF8E1TSLqIemGMxHhqOAYU4w6tvvXxM8stMgR5FNEEfMEMyKsp aIJUk09cHukVbU6DfVnT5Cf3aQF5GsjdZ/eOcxyuOX/l0DQzb6gIWvEo8gXIpTmTLQgM XxJFkgeyLDOO7j31eq9tzXXdKg1j2VijdD7t8xBYaf2ZAuWSskqhHMFwRuFp6PsMXksh dEog== X-Gm-Message-State: AOAM531yKetP2eg2c/baPYkrs0dvzZTWZJd290nC5Wrw26iCewa4bd0M aX102V4QH11yW5wVXPzL85o3dOeJnCWAhQti/TdJtyJYwLQ= X-Google-Smtp-Source: ABdhPJwryTnPVCaXTUtl1T9xuWKh1b2jfRCzL90RDLT7+NR9w5hXhxQz2cSZCymdZMShN/58TqzdyFi4506j7tsyj2k= X-Received: by 2002:a7b:c3cd:: with SMTP id t13mr7753723wmj.109.1615463334261; Thu, 11 Mar 2021 03:48:54 -0800 (PST) MIME-Version: 1.0 References: <40f8d9ec-2387-4e38-819d-fdb4407c5709@spark> <3089d87b-5884-44bf-8eb9-ef5d34f16756@spark> <23240094-4827-f85e-6ad5-6b22bc09905b@isode.com> In-Reply-To: From: Dave Cridland Date: Thu, 11 Mar 2021 11:48:43 +0000 Message-ID: To: Adrien de Croy Cc: Alexey Melnikov , "imapext@ietf.org" Content-Type: multipart/alternative; boundary="0000000000002fc5be05bd415ec9" Archived-At: Subject: Re: [imapext] SEARCHRES RFC5182 X-BeenThere: imapext@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of IMAP extensions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2021 11:48:59 -0000 --0000000000002fc5be05bd415ec9 Content-Type: text/plain; charset="UTF-8" On Wed, 10 Mar 2021 at 20:14, Adrien de Croy wrote: > Hi Alexey > > ------ Original Message ------ > From: "Alexey Melnikov" > To: "Adrien de Croy" > Cc: "imapext@ietf.org" > Sent: 10/03/2021 11:07:25 pm > Subject: Re: [imapext] SEARCHRES RFC5182 > > Hi Adrien, > On 09/03/2021 23:21, Adrien de Croy wrote: > > Hi Everyone > > I've been looking into this extension, but it has a number of problems and > uncertainties in the spec especially: > > * in relation to storing results of ESEARCH extensions, it refers to them > as messages). > > Can you elaborate/quote specific text which is confusing? > > > > When the SAVE result option is combined with the MIN or MAX [ESEARCH] > result option, and none of the other ESEARCH result options are > present, the corresponding MIN/MAX is returned (if the search result > is not empty), but the "$" marker would contain a single message as > returned in the MIN/MAX return item. > > > It's not clear if $ is storing the result of the MIN, or something else. > But I guess those are the same for MIN and MAX - the IDs. > > $ only stores message IDs, not messages. Maybe replace "contain a single > message" with "refers to a single message" or "contains the message ID" > > Well, it depends on how you see it conceptually. You could have the server store a list of UIDs always, and translate as required. You could also have the server store a list of tuples of (UID,seq), and filter as needed, updating the sequence numbers if an EXPUNGE occurs. Or you could have the server store $ as a std::vector> or something and format as UID or seq appropriately. In all three cases (and more) you're conceptually storing references to messages, though, and whether they're ids or not is somewhat irrelevant - when executing the search itself, you're working on messages rather than identifiers. The one thing that "$" definitely doesn't contain is a list of integers, though. > I guess I should have read it a few more times. Maybe it's not so unclear. > > > * in relation to reporting errors if the saved results are incompatible > (e.g. stored SEQ but asked to use as UID or vice versa) > > The document is quite clear that $ references message numbers or UIDs as > appropriate in the corresponding context. So a saved result is never > incompatible. > > > So we save both UIDs and SEQ and adjust these on expunge, and just use > whichever one is appropriate in the subsequent commands. > I suspect I'd store yet another internal identifier, myself, but loosely store whatever makes you get the right external behaviour. If UIDs work for you, I'd store only UIDs and treat a "$" as if it were "UID $" as needed, for example. But remember it's never a simple string substitution - otherwise "$" would fail if the saved set were to contain no messages. As for whether people use this in clients, I don't know - it's been fairly esoteric until now, but IMAP4rev2 might change this. It's very useful for "Mark all as" from a search output for online clients, but there aren't many of those around anymore. Dave. --0000000000002fc5be05bd415ec9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable

=
On Wed, 10 Mar 2021 at 20:14, Adrien = de Croy <adrien@qbik.com> wrot= e:
=20 =20
Hi Alexey

------ Or= iginal Message ------

From: "Alexey Melnikov" <alexey.melnikov@isode.com>

To: "Adrien de Croy" <adrien@qbik.com>

Cc: "imapex= t@ietf.org" <imapext@ietf.org>

Sent: 10/03/2021 11:07:25 pm

Subject: Re: [imapext] SEARCHRES RFC5182

Hi Adrien,

On 09/03/2021 23:21, Adrien de Croy wrote:

=20 =20 =20 =20 Hi Everyone

I've been looking into this extension, but it has a number o= f problems and uncertainties in the spec=C2=A0especially:

* in relation to storing results of ESEARCH extensions, it refers to them as messages).

Can you elaborate/quote specific text which is confusing?

When the SAVE result option = is combined with the MIN or MAX [ESEARCH]
=C2=A0 =C2=A0result option, an= d none of the other ESEARCH result options are
=C2=A0 =C2=A0present, the= corresponding MIN/MAX is returned (if the search result
=C2=A0 =C2=A0is= not empty), but the "$" marker would contain a single message as=
=C2=A0 =C2=A0returned in the MIN/MAX return item.

It's not clear if $ is storing the result of t= he MIN, or something else.=C2=A0 But I guess those are the same for MIN and= MAX - the IDs.

$ onl= y stores message IDs, not messages.=C2=A0 Maybe replace "contain a sin= gle message" with "refers to a single message" or "cont= ains the message ID"

Well, i= t depends on how you see it conceptually.

You coul= d have the server store a list of UIDs always, and translate as required.

You could also have the server store a list of tupl= es of (UID,seq), and filter as needed, updating the sequence numbers if an = EXPUNGE occurs.

Or you could have the server store= $ as a std::vector<std::shared_ptr<Message>> or something and = format as UID or seq appropriately.

In all three c= ases (and more) you're conceptually storing references to messages, tho= ugh, and whether they're ids or not is somewhat irrelevant - when execu= ting the search itself, you're working on messages rather than identifi= ers.

The one thing that "$" definitely d= oesn't contain is a list of integers, though.
=C2=A0
I guess I should have read it a few more times.=C2=A0 Maybe it's no= t so unclear.

* in relation to reporting errors if the saved results are incompatible (e.g. stored SEQ but asked to use as UID or vice versa)

The document is quite clear that $ references message numbers or UIDs as appropriate in the corresponding context. So a saved result is never incompatible.

So we save both UIDs and SEQ and adjust these on = expunge, and just use whichever one is appropriate in the subsequent comman= ds.

I suspect I'd sto= re yet another internal identifier, myself, but loosely store whatever make= s you get the right external behaviour. If UIDs work for you, I'd store= only UIDs and treat a "$" as if it were "UID $" as nee= ded,=C2=A0for example. But remember it's never a simple string substitu= tion - otherwise "$" would fail if the saved set were to contain = no messages.

As for whether people use this in cli= ents, I don't know - it's been fairly esoteric until now, but IMAP4= rev2 might change this. It's very useful for "Mark all as" fr= om a search output for online clients, but there aren't many of those a= round anymore.

Dave.