Subject: Re: [regext] rfc7484bis

From: Gavin Brown <[email-masked]>
Date: Tue, 4 Aug 2020 20:32:47 +0100

Hi Marc,

> as Scott is updating RFC7482,RFC7483 for standard level, I’m doing the same for rfc7484. I haven’t heard major issues or major fixes to be made for rfc7484. I have a few wording fixes only at this time. There were some discussions on enhancing RFC7484 for other use cases, but never went far.
> if anyone has a something to raise for RFC7484, please send me email asap.

RFC 7484 doesn't provide any guidance for client implementers about how to select a base RDAP URL when the services array contains more than one entry. As a result I suspect that in this scenario different implementations will behave in different ways, and users are at risk of seeing different responses depending on the client they use.

My suggestions are that:-

1. client implementers should be advised to prefer https:// base URLs over http:// base URLs.

2. server operators should be advised that if multiple base URLs with the same scheme are present in an entry, then all the RDAP endpoints referenced by these base URLs must return identical responses (for the same RDAP query).

Thanks,

G.

--
Gavin Brown
Head of Registry Services and Chief Innovation Officer
CentralNic Group plc (LSE:CNIC)
https://www.centralnic.com

Tel: +44.7548243029

CentralNic Group plc is a company registered in England and Wales with company number 8576358. Registered Offices: Saddlers House, Gutter Lane, London EC2V 6AE.

---

From: "Patrick Mevzek" <[email-masked]>
Date: Tue, 04 Aug 2020 14:47:31 -0500

> 1. client implementers should be advised to prefer https:// base URLs 

I think this is already addressed by this text in the current RFC:
"
   Per [RFC7258], in each array of base RDAP URLs, the secure versions
   of the transport protocol SHOULD be preferred and tried first.  For
   example, if the base RDAP URLs array contains both HTTPS and HTTP
   URLs, the bootstrap client SHOULD try the HTTPS version first.
"
 
> 2. server operators should be advised that if multiple base URLs with 
> the same scheme are present in an entry, then all the RDAP endpoints 
> referenced by these base URLs must return identical responses (for the 
> same RDAP query).

Why "with the same scheme" here? If there is both `http://` and `https://`,
even if that is not advised, shouldn't both cases respond the same way?

PS: related but not directly, at least for domain registries, it would be
nice to have an `SRV` record on `_rdap._tcp` or something to point to relevant
RDAP server, even if that does not allow to encode the path (but maybe a solution with .well-known/ and URI template could be found), it allows at least
for nice failover and load balancing. It may be a problem for gTLDs as they have
restrictions in content of their zone.
Maybe the newly expected SCVB record could help...

A setup like that would allow for discoverability without centralization of data,
which also removes IANA from the hot operational path when RDAP clients do queries.

-- 
  Patrick Mevzek
  [email-masked]

---

From: "Marc Blanchet" <[email-masked]>
Date: Tue, 04 Aug 2020 16:57:57 -0400

On 4 Aug 2020, at 15:32, Gavin Brown wrote:

> Hi Marc,
>
>> as Scott is updating RFC7482,RFC7483 for standard level, I’m doing 
>> the same for rfc7484. I haven’t heard major issues or major fixes 
>> to be made for rfc7484. I have a few wording fixes only at this time. 
>> There were some discussions on enhancing RFC7484 for other use cases, 
>> but never went far.
> RFC 7484 doesn't provide any guidance for client implementers about 
> how to select a base RDAP URL when the services array contains more 
> than one entry. As a result I suspect that in this scenario different 
> implementations will behave in different ways, and users are at risk 
> of seeing different responses depending on the client they use.
>

good points. agreed. Will add.

Marc.

> CentralNic Group plc is a company registered in England and Wales with 
> company number 8576358. Registered Offices: Saddlers House, Gutter 
> Lane, London EC2V 6AE.

---

From: "Marc Blanchet" <[email-masked]>
Date: Tue, 11 Aug 2020 14:33:38 -0400

>    URLs, the bootstrap client SHOULD try the HTTPS version first.

Gavin,
 Patrick is right. text was already there. ;-) Happy with the current text?

Marc.

---

From: "Marc Blanchet" <[email-masked]>
Date: Tue, 11 Aug 2020 14:55:17 -0400

On 4 Aug 2020, at 15:47, Patrick Mevzek wrote:
> RDAP server, even if that does not allow to encode the path (but maybe 
> a solution with .well-known/ and URI template could be found), it 
> allows at least

well, this has been debated at length during the WEIRDS working group 
work. I actually wrote a sentence about this in the RFC (in the 
acknowledgements section). I’m not sure we want to restart the debate 
again…

> which also removes IANA from the hot operational path when RDAP 

yes. this is the well-known caveat of this RFC and discussed and debated 
during WEIRDS. But experience up to now has not shown any issue, at 
least to my knowledge. (and as a developer of the RDAP Browser mobile 
app, I haven’t seen any issue fetching that registry. I do have found 
thousands of issues with the registry/registrars RDAP servers however, 
but that is another story).

Marc.

---

From: "Patrick Mevzek" <[email-masked]>
Date: Tue, 11 Aug 2020 14:27:59 -0500

Hello Marc,

> acknowledgements section). I’m not sure we want to restart the debate 

I am not saying to restart the debate, especially not in the context of a -bis document where protocol changes are not welcome.

But the RFCs are also 5 years old now and a lot of things change quickly.
SVCB record in the DNS being one, while not there already.
 
> but that is another story).

I am not saying there is a current issue, fetching the JSON file from IANA
webserver is clearly the smallest problem of any RDAP client.

But I also think there is currently no issue because basically the world did
not shift to RDAP in any way yet. Which can easily be witnessed by the amount
of broken servers so far - even if they are in a regulated space where compliance
is an issue - and the total lack of ccTLDs in this space, at least in operations.

Once there is a shift, then an issue can happen. Or not. We can indeed not
try to prematurely optimize this.

And I would prefer a more decentralized way of discovery, because it makes
more sense to me and because we do already have a decentralized publicly available
database (the DNS) that could store RDAP related information per registry.

-- 
  Patrick Mevzek
  [email-masked]

---

From: Gavin Brown <[email-masked]>
Date: Wed, 12 Aug 2020 16:25:18 +0100

> On 11 Aug 2020, at 19:33, Marc Blanchet <[email-masked]> wrote:
> 

No issues here! My second point (about identical responses from every listed server) still stands though.

Here's my proposed text, which could be inserted below the aforementioned paragraph in Section 3:

"Registrants of entries in bootstrap registries SHOULD ensure, where multiple base URLs are listed for a given set of entries, that all listed RDAP servers produce the same response to a given query."

More than happy to accept feedback on the above.

G.

Tel: +44.7548243029

---

From: "Marc Blanchet" <[email-masked]>
Date: Wed, 12 Aug 2020 12:18:26 -0400

On 12 Aug 2020, at 11:25, Gavin Brown wrote:

>>>> 1. client implementers should be advised to prefer https:// base 
>>>> URLs
>>>> over http:// base URLs.
>>>
> No issues here!

ok.

> My second point (about identical responses from every listed server) 
> still stands though.
>

well, already added text in the published draft yesterday.

https://www.ietf.org/internet-drafts/draft-blanchet-regext-rfc7484bis-00.txt

extract of my added text:
« All RDAP endpoints referenced by the URLs in the array MUST return
    identical responses for the same RDAP query. »

would that work?

Marc.

> "Registrants of entries in bootstrap registries SHOULD ensure, where 
> multiple base URLs are listed for a given set of entries, that all 
> listed RDAP servers produce the same response to a given query."
>

---

From: "Marc Blanchet" <[email-masked]>
Date: Wed, 12 Aug 2020 12:22:36 -0400

On 11 Aug 2020, at 15:27, Patrick Mevzek wrote:

>>> PS: related but not directly, at least for domain registries, it 
>>> would
>>> be
>>> nice to have an `SRV` record on `_rdap._tcp` or something to point 
>>> to
>>> relevant
>>> for nice failover and load balancing. It may be a problem for gTLDs 
>>> as
>>> they have
> I am not saying to restart the debate, especially not in the context 
> of a -bis document where protocol changes are not welcome.
>

we all wanted. But when one think of all the requirements and that we 
make sure it works for both domains and IP addresses and AS numbers, the 
best compromise solution was the IANA registries. None of the solutions 
discussed were perfect. Yes, the IANA registry solution is not 
distributed, but brings additional features (as agile as we want with a 
JSON data model) that were not available in in-dns solutions, available, 
ready to deploy, and also, one of the point that was pushed often was 
the ability to fetch the data from a contained Javascript client in a 
browser (where dns queries are not available).

Marc.

---

From: Gavin Brown <[email-masked]>
Date: Thu, 13 Aug 2020 12:39:54 +0100

> On 12 Aug 2020, at 17:18, Marc Blanchet <[email-masked]> wrote:
> 

Looks good to me.

G.

Tel: +44.7548243029

---

From: George Michaelson <[email-masked]>
Date: Mon, 17 Aug 2020 09:27:27 +1000

I would like to see some allowance of limited metadata about which
node responded, or other information variance. If thats confined to
comments in the outer protocol, it means the response inline cannot be
used to debug specific problems with the node which responded.

We see this all the time in DNS: The lack of information about the
service methods used to supply a response makes debug significantly
harder.

in-band introspection is useful. It isn't about the response, its
about how the response came to be.

-G

On Thu, Aug 13, 2020 at 9:40 PM Gavin Brown <[email-masked]> wrote:
> _______________________________________________
> regext mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/regext

---

From: Rubens Kuhl <[email-masked]>
Date: Sun, 16 Aug 2020 23:08:10 -0300

> On 11 Aug 2020, at 16:27, Patrick Mevzek <[email-masked]> wrote:
> 
> Hello Marc,
> 

Of the 822 https:// URLs in https://data.iana.org/rdap/dns.json <https://data.iana.org/rdap/dns.json> , and 0 http:// URLs, there are some for ccTLDs:
.ar
.br
.ca
.cr
.cz
.id
.is
.no
.tz

So, I believe we could remove http:// as a transport option, and that there is no total lack of ccTLDs. Surely there could be more.

Rubens

---

From: "Patrick Mevzek" <[email-masked]>
Date: Mon, 17 Aug 2020 01:17:08 -0500

On Sun, Aug 16, 2020, at 21:08, Rubens Kuhl wrote:
> Of the 822 https:// URLs in https://data.iana.org/rdap/dns.json , and 0 
> http:// URLs, there are some for ccTLDs:
> .ar
> .br
> .ca
> .cr
> ..cz
> .id
> .is
> .no
> .tz
> 
> 
> So, I believe we could remove http:// as a transport option, and that 
> there is no total lack of ccTLDs. Surely there could be more. 

Existence in the bootstrap does not guarantee it works, neither that it
respects the specifications. But ok,  replace "total lack" by "near total lack"
(considering the number of ccTLDs out there...)

$ curl -s https://rdap.nic.ar/domain/nic.ar | jq .rdapConformance
[
  "rdap_level_0",
  "fred_version_0"
]

$ curl -s https://rdap.registro.br/domain/registro.br | jq .rdapConformance
[
  "rdap_level_0",
  "nicbr_level_0"
]

$ curl -s https://rdap.ca.fury.ca/rdap/domain/fury.ca | jq .rdapConformance
[
  "icann_rdap_response_profile_0",
  "icann_rdap_technical_implementation_guide_0",
  "rdap_level_0"
]

$ curl -s https://rdap.nic.cr/domain/nic.cr | jq .rdapConformance
[
  "rdap_level_0",
  "fred_version_0"
]

$ curl -s https://rdap.nic.cz/domain/nic.cz | jq .rdapConformance
[
  "rdap_level_0",
  "fred_version_0"
]

$ curl -s https://rdap.pandi.id/rdap/domain/pandi.id | jq .rdapConformance
[
  "icann_rdap_technical_implementation_guide_0",
  "rdap_level_0",
  "rdap_pandi_level_0"
]

$ curl -s https://rdap.isnic.is/rdap/domain/isnic.is | jq .rdapConformance
[
  "rdap_level_0",
  "isnic_level_0"
]

$ curl -s https://rdap.norid.no/domain/norid.no | jq .rdapConformance
[
  "rdap_level_0",
  "rdap_objectTag_level_0",
  "norid_level_0"
]

$ curl -s https://whois.tznic.or.tz/rdap/domain/tznic.or.tz | jq .rdapConformance
[
  "rdap_level_0",
  "fred_version_0"
]

Please compare to the current list of the registered RDAP namespaces at https://www.iana.org/assignments/rdap-extensions/rdap-extensions.xhtml#rdap-extensions-1

- arin_originas0
- artRecord
- cidr0
- fred
- icann_rdap_response_profile_0
- icann_rdap_technical_implementation_guide_0
- platformNS
- rdap_objectTag
- regType

It would be very good hence to see there also registered all the ones
existing in the wild...
That would be a step better than just being listed in the bootstrap file,
at least for any client implementer.

-- 
  Patrick Mevzek
  [email-masked]

---

From: "Marc Blanchet" <[email-masked]>
Date: Fri, 21 Aug 2020 12:27:54 -0400

> about how the response came to be.

I get your point. Can you provide some text or some proposal on that 
matter?

Marc.

>> CentralNic Group plc is a company registered in England and Wales 
>> with company number 8576358. Registered Offices: Saddlers House, 
>> Gutter Lane, London EC2V 6AE.
>>

---

From: George Michaelson <[email-masked]>
Date: Mon, 24 Aug 2020 16:31:24 +1000

How about this:

----
All RDAP endpoints referenced by the URLs in the array MUST return
identical responses for the same RDAP query, except that the “notices”
data structure MAY contain an object informing the client of the
identity of the endpoint. If such an object is provided it SHOULD use
the registered notices and remarks type value of “service node
identification” and it SHOULD be returned by all endpoints.

The “notices” structure is only permitted at the top level of a
response and contains information about the service or response rather
than about the object involved.
----

Which requires in IANA considerations:

----
 IANA is requested to add that value to the
https://www.iana.org/assignments/rdap-json-values/rdap-json-values.xhtml
registry.
----

My colleague, who suggested these words, added

"The places to bike shed is over whether an endpoint MAY or SHOULD
return that object, and whether it SHOULD or MUST have a fixed value,
and whether all endpoints SHOULD or MUST return the object if any do."

I think on the whole, thats a might fine list of bike sheds. I don't
know the answer to any of them. I tend to MAY return, but SHOULD be a
consistent (if not fixed) value, and all endpoints MUST return the
object if any do.

My reasoning is. that the debug utility is only present if they all do
it, so it is all or none. But, since it is not always needed, it can
be turned off and on which goes to MAY. Consistency is key; the
variance should be to identify a node, amongst all nodes.  Or software
version. But, definitionally that cannot be "identical" because.. well
you might have upgraded only one node. So consistent, is not SHOULD or
MUST be a fixed value.

-G