<?xml version="1.0" encoding="utf-8"?>
<?xml-model href="rfc7991bis.rnc"?>  <!-- Required for schema validation and schema-aware editing -->
<!-- <?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?> -->
<!-- This third-party XSLT can be enabled for direct transformations in XML processors, including most browsers -->


<!DOCTYPE rfc [
  <!ENTITY nbsp    "&#160;">
  <!ENTITY zwsp   "&#8203;">
  <!ENTITY nbhy   "&#8209;">
  <!ENTITY wj     "&#8288;">
]>
<!-- If further character entities are required then they should be added to the DOCTYPE above.
     Use of an external entity file is not recommended. -->

<rfc
  xmlns:xi="http://www.w3.org/2001/XInclude"
  category="info"
  docName="draft-hu-6man-ipv6-flowlabel-load-balancing-rdma-04"
  ipr="trust200902"
  obsoletes=""
  updates=""
  submissionType="IETF"
  xml:lang="en"
  version="3">

  <front>
    <title abbrev="draft-hu-6man-ipv6-flowlabel-load-balancing-rdma-04"> A RoCEv2 Flow-Level Load Balancing Method Based
      on the IPv6 Flow Label
    </title>
    <!--  [REPLACE/DELETE] abbrev. The abbreviated title is required if the full title is longer than 39 characters -->

    <seriesInfo name="Internet-Draft" value="draft-hu-6man-ipv6-flowlabel-load-balancing-rdma-04"/>
   
    <author fullname="Jiayuan Hu" initials="Jiayuan" surname="Hu">
      <organization>China Telecom</organization>
      <address>
        <postal>
          <street>109, West Zhongshan Road, Tianhe District</street>
          <city>Guangzhou</city>
          <region>Guangzhou</region>
          <code>510000</code>
          <country>CN</country>
        </postal>
        <email>hujy5@chinatelecom.cn</email>
      </address>
    </author>

    <author fullname="Jie Dong" initials="J." surname="Dong">
      <organization>Huawei Technologies</organization>
      <address>
        <postal>
          <street>Huawei Bld., No.156 Beiqing Rd.</street>
          <city>Beijing</city>
          <code>100095</code>
          <country>CN</country>
        </postal>
        <email>jie.dong@huawei.com</email>
      </address>
    </author>

     <author fullname="Yao Liu" surname="Liu">
      <organization>ZTE Corporation</organization>
      <address>
        <postal>
          <street/>
          <!-- Reorder these if your country does things differently -->
         <city></city>
          <region/>
          <code/>
          <country>China</country>
        </postal>
        <phone></phone>
        <email>liu.yao71@zte.com.cn</email>
        <!-- uri and facsimile elements may also be added -->
     </address>
    </author>

    <author fullname="Xia Gong" initials="Xia" surname="Gong">
      <organization>China Telecom</organization>
      <address>
        <postal>
          <street>109, West Zhongshan Road, Tianhe District</street>
          <city>Guangzhou</city>
          <region>Guangzhou</region>
          <code>510000</code>
          <country>CN</country>
        </postal>
        <email>gongxia@chinatelecom.cn</email>
      </address>
    </author>

    <date year="2026"/>

    <area>Internet</area>
    <workgroup>IPv6 Maintenance</workgroup>
    <!-- "Internet Engineering Task Force" is fine for individual submissions.  If this element is 
          not present, the default is "Network Working Group", which is used by the RFC Editor as 
          a nod to the history of the RFC Series. -->

    <keyword>RFC</keyword>
    <!-- [REPLACE/DELETE]. Multiple allowed.  Keywords are incorporated into HTML output files for 
         use by search engines. -->

    <abstract>
      <t>
        This document proposes a method for achieving flow-level load balancing in RoCEv2 (RDMA over Converged Ethernet
        version 2) networks. Traditional per-flow load balancing based on the 5-tuple cannot distinguish between
        different RDMA sessions that share the same 5-tuple. This causes "elephant flows" to be hashed to the same
        path, leading to network congestion. This method resolves this issue by having the ingress network device (e.g.,
        a data center gateway or DCI edge router) parse the QP (Queue Pair) information
        from the IB BTH (Base Transport Header) and IB DETH (Datagram Extended Transport Header, when present)
        of the RoCEv2 packet. By combining this with portions of the IPv6 source and destination addresses as an entropy source, a
        CRC32 hash algorithm generates a 32-bit value, the low 20 bits of which are then written into the Flow
        Label field of the IPv6 header. Network devices can subsequently use the updated "5-tuple + Flow Label" for more granular flow-level
        load balancing, thereby effectively improving transmission efficiency in high-performance networks for
        workloads such as AI computing.
      </t>
    </abstract>
 
  </front>


  <middle>
    
    <section>
      <name>Introduction</name>
      <t>
        The rapid advancement of Artificial Intelligence (AI) and High-Performance Computing (HPC) has driven the
        widespread adoption of Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCEv2) in data center and
        intelligent computing networks, and increasingly across data center interconnect (DCI) links that connect
        geographically dispersed intelligent computing data centers. RoCEv2 enables high-throughput, low-latency data transfers that are critical
        for distributed training and storage workloads. However, the effective operation of these networks is challenged
        by the inherent characteristics of RDMA traffic, particularly the "elephant flow" problem.
      </t>
      <t>
        Traditional load balancing mechanisms in IP networks typically rely on a 5-tuple (source/destination IP address,
        source/destination port, and protocol number) to identify and distribute traffic flows. In RoCEv2 networks, a
        significant limitation arises: multiple distinct RDMA sessions or flows generated by the same upper-layer
        application may share an identical 5-tuple. This is because the RDMA Queue Pair (QP) information, which uniquely
        identifies a session, is encapsulated within the InfiniBand Base Transport Header (IB BTH) and the optional Datagram
        Extended Transport Header (IB DETH) of the RoCEv2 packet. Consequently, conventional 5-tuple-based hashing treats these
        distinct RDMA flows as a single entity and forwards them to the same network path, leading to severe congestion,
        packet loss, and a significant degradation in overall network throughput and performance.
      </t>
      <t>
        It should be noted that the identical-5-tuple condition is most relevant when the UDP source port is fixed or
        stable across sessions of the same application; if the source port were randomized per session, the 5-tuple
        alone could already distinguish some flows. However, relying on UDP source-port randomization is not always
        available or desirable (for example, it may conflict with established port conventions, firewall or NAT
        policies, or the transport stack's port-selection behavior), and it does not change the fundamental value of
        exposing the QP as a direct, stable entropy source. This document therefore focuses on deriving the Flow
        Label from the QP, which is carried in every RoCEv2 packet regardless of the UDP source port.
      </t>
      <t>
        To address this problem, this document introduces a method for flow-level load balancing that leverages the
        IPv6 Flow Label. The core idea is to enable an ingress network device (e.g., a data center gateway
        or DCI edge router at the DCI / WAN edge) to extract the QP pair information (source QP and destination QP)
        from the RoCEv2 packets. This extracted QP pair information is then used as input to a CRC32-based hash function
        to generate a per-flow identifier that is unique with high probability. This identifier is subsequently mapped into the Flow Label field of
        the IPv6 header. The sending host is not required to set the Flow Label; it may send packets with the Flow Label
        set to zero, as permitted by <xref target="RFC6437"/>.
      </t>
      <t>
        By combining the traditional 5-tuple with this dynamically generated Flow Label, the proposed method creates a
        fine-grained "5-tuple + Flow Label" flow identification key. This allows network devices to effectively
        distinguish between different RDMA sessions that were previously indistinguishable, thereby achieving true
        flow-level load balancing. This approach minimizes path collisions, reduces congestion, and enhances the
        utilization of multi-path DCI topologies, i.e., the parallel equal-cost links that connect the source and
        destination data centers.
      </t>
      <t>
        This document outlines the concept, details the packet processing method, and describes the mapping of the QP
        pair to the IPv6 Flow Label field. The subsequent sections will cover the mechanism in detail, discuss its
        advantages over existing solutions, and describe its applicability to RoCEv2 traffic carried across data
        center interconnects.
      </t>
      <t>
        The mechanism applies where RoCEv2 packets are carried natively over IPv6 on the DCI links, i.e., where
        the IPv6 header carrying the relabeled Flow Label is the header on which DCI load balancers hash. If
        the DCI instead encapsulates traffic behind an outer tunnel header (e.g., VXLAN or L2 VPN
        encapsulations), Equal-Cost Multi-Path (ECMP) hashing is performed on the outer header, and
        relabeling the inner IPv6 Flow Label alone would not influence path selection.
      </t>
    </section>
      
    <section title="Conventions Used in This Document">
      <section>
        <name>Requirements Language</name>
        <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL",
          "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT
          RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
          interpreted as described in BCP 14 <xref target="RFC2119"/>
          <xref target="RFC8174"/> when, and only when, they appear in
          all capitals, as shown here.</t>
      </section>

      <section title="Abbreviations and Terminology">
        <t> RoCEv2: RDMA over Converged Ethernet version 2</t>
        <t> RDMA: Remote Direct Memory Access</t>
        <t> QP: Queue Pair</t>
        <t> IB BTH: InfiniBand Base Transport Header</t>
        <t> IB DETH: InfiniBand Datagram Extended Transport Header</t>
        <t> Flow Label: The 20-bit field in the IPv6 header used for flow identification, as defined in
          <xref target="RFC6437"/>.</t>
        <t> UD: Unreliable Datagram, an InfiniBand transport service in which the IB DETH is present.</t>
        <t> RC: Reliable Connection, an InfiniBand transport service in which the IB DETH is absent.</t>
        <t> CRC32: Cyclic Redundancy Check 32-bit algorithm.</t>
        <t> CRC-32/ISO-HDLC: The specific CRC-32 variant used as the hash in this document, i.e., the
          standard CRC-32 defined by generator polynomial 0x04C11DB7, initial register value 0xFFFFFFFF,
          reflected (LSB-first) bit processing, and a final XOR with 0xFFFFFFFF. This is the variant used by
          Ethernet, zlib/gzip, and many other protocols.</t>
        <t> PRNG: Pseudo-Random Number Generator</t>
        <t> Elephant Flow: A data flow that transfers a large volume of data (e.g., gigabytes or more)
          over a sustained period, often associated with distributed training, checkpointing, and parameter
          synchronization in AI/HPC workloads. </t>
      </section>
    </section>
      <!-- [CHECK] The 'Requirements Language' section is optional -->

    <section>
      <name>Relation to Existing Standards and the Need for QP-based Entropy</name>
      <t>
        This method extends the principles established in <xref target="RFC6437"/> and <xref target="RFC6438"/> for
        using the IPv6 Flow Label. <xref target="RFC6437"/> recommends source hosts set the Flow Label using a
        PRNG to provide entropy for load balancing. <xref target="RFC6438"/> further specifies that the tunnel
        ingress can set the Flow Label of the encapsulated packets, so that ECMP hashing of the tunneled
        traffic gains entropy from the encapsulated flow.
      </t>
      <t>
        <xref target="RFC7098"/> describes using the IPv6 Flow Label for layer 3/4 load balancing in large server farms,
        suggesting that {source address, flow label} can serve as a session key when the Flow Label is non-zero. The
        present document complements <xref target="RFC7098"/> in two key aspects: 1. <xref target="RFC7098"/> targets
        general HTTP/TCP traffic where the 5-tuple is sufficient to distinguish sessions, whereas this document addresses
        RoCEv2 traffic carried over DCI links, where multiple RDMA flows may share an identical 5-tuple (when the UDP source port is fixed); 2. <xref target="RFC7098"/> assumes the
        source host sets the Flow Label per <xref target="RFC6437"/>, while this document specifies that the ingress
        network device writes the Flow Label based on QP information from the RoCEv2 payload, with the host setting it
        to zero as a signal for relabeling.
      </t>
      <t>
        However, these standards share a common model: they assume that the entity setting the Flow Label has
        access to the transport-layer identity of each flow. <xref target="RFC6437"/> places this responsibility
        primarily on the source host, recommending that each transport connection or flow be assigned its
        own distinct label, <xref target="RFC6438"/> delegates it to the tunnel ingress endpoint, and
        <xref target="RFC7098"/> assumes
        the source host has already set the label. In RoCEv2 deployments this model is not realized in practice:
        RDMA host stacks typically emit packets with the Flow Label set to zero, so the host-side per-flow
        labeling these standards recommend is absent. Furthermore, when the UDP source port is fixed (as discussed
        in the Introduction), the flows that need to be distinguished are not distinguishable at the network
        layer at all: a single host process (e.g., a GPU communicator library) often creates multiple parallel
        RDMA sessions that share the same source and destination IPs (typically the host interface IPs) and port
        numbers (destination UDP port 4791, the port assigned to RoCEv2 by IANA), and the only per-session identifiers
        exposed to the network are the QP numbers carried inside the RDMA transport headers.
      </t>
      <t>
        In this scenario, a standard PRNG has no knowledge of the internal RDMA session structure: even when
        applied by the host stack, it operates at the flow granularity visible to the network (the
        5-tuple), so packets of concurrent sessions sharing that 5-tuple would all receive the same
        Flow Label. Randomizing the label per packet instead would produce values that cannot be
        correlated with the actual sessions and would break in-order delivery within a session.
        Either way, the "elephant flow" problem would not be solved.
      </t>
      <t>
        To overcome this, the proposed method extracts the actual source of session differentiation: the Queue Pair (QP)
        numbers. QP numbers uniquely identify RDMA sessions. By hashing this information, the
        Flow Label becomes a direct function of the session
        itself, not just the network 5-tuple.
      </t>
      <t>
        Moreover, to ensure compatibility with the expectations of <xref target="RFC6437"/>, particularly Section 2, the following
        operational rules are established for this scheme:
      </t>
      <t>
        1. As specified in <xref target="RFC6437"/>, a Flow Label of zero is used to indicate packets that have not been explicitly labeled.
        Hosts participating in this scheme SHOULD either set the Flow Label to zero or not set it at all (which results
        in a default value of zero). This zero value serves as a signal to the network that the packet is eligible for
        relabeling by the ingress network device.
      </t>
      <t>
        2. If the arriving packet already carries a non-zero Flow Label value (e.g., set by the sending host or by
        another upstream device), the ingress network device MUST NOT overwrite it. This rule aligns with the core
        principle of <xref target="RFC6437"/> that "once set to a non-zero value, the Flow Label is expected to be
        delivered unchanged to the destination node(s)."
      </t>
      <t>
        3. Per <xref target="RFC6437"/> Section 3, the capability of the ingress network device to set the Flow Label
        based on QP information MUST be implemented as a configurable option, disabled by default. This ensures that
        the feature is under the control of network managers, avoiding unwanted per-packet processing load on the
        ingress device and any other undesirable effects. Network operators MAY enable this feature on ports where
        RoCEv2 traffic is expected.
      </t>
      <t>
        Note that if a source host does set per-flow labels according to <xref target="RFC6437"/>, rule 2 above
        preserves them, and such host-assigned randomized labels themselves already provide flow-level
        spreading. The relabeling mechanism in this document therefore targets the common deployment case in
        which RoCEv2 host stacks emit packets with the Flow Label set to zero, where the network cannot rely on
        the host to provide any per-flow entropy.
      </t>
    </section>

    <section title="Flow-Level Load Balancing Based on the IPv6 Flow Label">
      <section>
        <name>Construction of the Hash Input</name>
        <t>
          Ensuring the generated Flow Label can uniquely identify an RDMA flow with high probability while
          possessing sufficient randomness to minimize collision probability is critical. The procedure for constructing the hash input is as follows:
        </t>
        <t>
          1. Extract the QP pair:
        </t>
        <t>
          * Dst_QP: Extracted from the IB BTH (Base Transport Header), 24 bits long (e.g., 0xABCDEF). The IB BTH
          is mandatory and always present in the RoCEv2 packet, so Dst_QP is always available.
        </t>
        <t>
          * Src_QP: If the IB DETH (Datagram Extended Transport Header) is present, extract Src_QP from the DETH, 24
          bits long (e.g., 0x123456). The IB DETH is optional and only present in UD service.
          In RC and other transport services commonly used in AI computing networks, the
          DETH is absent, and consequently Src_QP is not available.
        </t>
        <t>
          2. Generate the Entropy Source:
        </t>
        <t>
          To increase hash randomness, an entropy source is introduced. This scheme recommends using portions of the
          IPv6 addresses.
        </t>
        <t>
            * Take the lower 16 bits of the IPv6 source address as the first entropy source, Entropy_Src.
        </t>
        <t>
            * Take the lower 16 bits of the IPv6 destination address as the second entropy source, Entropy_Dst.
        </t>
        <t>
          When DETH is present (Src_QP available): Hash_Input = {Src_QP || Dst_QP || Entropy_Src || Entropy_Dst} (80 bits)
        </t>
        <t>
          When DETH is absent (Src_QP not available): Hash_Input = {Dst_QP || Entropy_Src || Entropy_Dst} (56 bits)
        </t>
        <t>
          In the latter case, since only Dst_QP is used as the QP entropy source, the overall entropy is reduced.
          However, this is a reasonable trade-off: Dst_QP alone is still sufficient to distinguish concurrent RDMA
          flows between the same pair of hosts, because the sessions of a receiver are uniquely identified by
          their destination QP numbers, and for such flows the address entropy is necessarily identical anyway.
          The role of the address entropy is therefore complementary: it differentiates flows whose QP numbers
          happen to coincide across different host pairs, which benefits load balancers that hash on the Flow
          Label alone or on an address-plus-Flow-Label key (as suggested in <xref target="RFC7098"/>) rather
          than on the full 5-tuple.
        </t>
        <t>
          All fields in the Hash_Input are concatenated in network byte order (most significant byte first),
          as illustrated by the byte-wise example in the next subsection. The computation MUST be
          deterministic: an implementation MUST apply exactly the procedure specified in this document, so
          that any device that relabels packets of the same RDMA session in the same direction (e.g., primary
          and backup DCI edge devices) derives the same Flow Label for the same Hash_Input.
        </t>
      </section>
        <section>
          <name>Hash by CRC32 Algorithm</name>
          <t>
            This document uses the standard CRC-32 algorithm, i.e., the CRC-32/ISO-HDLC variant (generator
            polynomial 0x04C11DB7, initial register value 0xFFFFFFFF, reflected bit processing, and final XOR
            with 0xFFFFFFFF), as the core hash. This is the well-known CRC-32 formulation used by Ethernet and
            by widely deployed software libraries such as zlib, ensuring identical results across
            implementations. CRC32 offers advantages such as fast computation, hardware-friendly implementation,
            and a low collision rate, making it highly suitable for line-rate forwarding in network devices.
          </t>
          <t>
            The first step is a byte-wise split (using Hash_Input = 0x123456ABCDEF00010002): 0x12, 0x34, 0x56, 0xAB, 0xCD, 0xEF, 0x00, 0x01, 0x00, 0x02
          </t>
          <t>
            The second step is iterative processing per byte (using the first byte 0x12 as an example):
          </t>
          <t>
            Step 1 (XOR): XOR the least significant 8 bits of the CRC register with the byte 0x12.
          </t>
          <t>
            Step 2 (8-bit Shift-XOR Loop): Process the result from Step 1 bit-by-bit for 8 iterations. In each iteration:
          </t>
          <t>
            a. Check the least significant bit (LSB) of the CRC register.
          </t>
          <t>
            b. Shift the CRC register right by one bit (pad the high bit with 0).
          </t>
          <t>
            c. If the LSB was 1, XOR the result with the reflected polynomial 0xEDB88320 (the bit-reversed form of the generator polynomial 0x04C11DB7).
          </t>
          <t>
            Repeat Steps 1 and 2 for all subsequent bytes.
          </t>
          <t>
            After processing all bytes, XOR the CRC register with 0xFFFFFFFF to obtain the final 32-bit hash
            result. For the example Hash_Input above (0x123456ABCDEF00010002), the resulting 32-bit value is 0x8DD783D7.
          </t>
        </section>
        <section>
          <name>Flow Label Field Population</name>
          <t>
            From the 32-bit CRC32 hash result, take the lower 20 bits as the Flow Label value and write this 20-bit
            value into the Flow Label field of the IPv6 header.
          </t>
          <figure>
          <name>Updated IPv6 Header Structure Showing the Newly Populated Flow Label Field</name>
          <artwork align="center"><![CDATA[
+---------+---------+---------+---------+---------+---------+---------+
| Version | Traffic Class     |          Flow Label (20 bits)         |
+---------+---------+---------+---------+---------+---------+---------+
|         Payload Length      |     Next Header   |     Hop Limit     |
+---------+---------+---------+---------+---------+---------+---------+
|                                                                     |
+                       IPv6 Source Address                           +
|                                                                     |
+---------+---------+---------+---------+---------+---------+---------+
|                                                                     |
+                       IPv6 Destination Address                      +
|                                                                     |
+---------+---------+---------+---------+---------+---------+---------+
|   ...   |      UDP Header   |   ...   | IB BTH  |  IB DETH (opt.)   |
+---------+---------+---------+---------+---------+---------+---------+
            ]]>
          </artwork>
        </figure>
        <t>
          Because the Flow Label is a deterministic function of the QP numbers and the address entropy, every
          packet of a given RDMA session in a given direction receives the same label. A load balancer that
          includes the Flow Label in its hash key therefore maps all packets of one session in that
          direction onto a single path, preserving in-order delivery within the session. This property
          is essential for RoCEv2
          Reliable Connection traffic, which depends on in-order delivery per Queue Pair (out-of-order
          arrival would trigger expensive go-back-N retransmissions), and it is consistent with the general
          rule of <xref target="RFC6437"/> that packet flows should not be reordered. The Flow Label adds
          entropy between sessions; it never spreads the packets of one session across multiple paths,
          except for the IPv6 fragmentation case discussed below.
        </t>
        </section>
        <section>
          <name>Where the Flow Label is Modified</name>
          <t>
            In this method, the Flow Label field is modified by the ingress network device (e.g., a data center gateway or
            DCI edge router at the DCI / WAN edge), not by the sending host. The sending host sends the RoCEv2 packet with the IPv6 Flow Label set to zero or
            leaves it unset, so that the packet is eligible for relabeling (see the operational rules above).
            Upon receiving the packet, the ingress network device parses the QP information from the IB BTH
            and, when present, the IB DETH; it then computes the CRC32 hash and writes the resulting 20-bit value
            into the Flow Label field before forwarding the packet toward the destination.
          </t>
          <t>
            Because RDMA sessions are bidirectional, the same relabeling is performed in the reverse direction by the
            ingress network device at the DCI / WAN edge of the destination data center. Return-direction packets
            (e.g., acknowledgements and read responses) carry the destination QP (Dst_QP) in their IB BTH, so the
            reverse-direction device can derive the corresponding Flow Label from that QP together with the address
            entropy and write it before forwarding the packet toward the source. Note that in the RC
            service the IB DETH is absent and only Dst_QP is available, so the reverse-direction
            hash input falls back to the same Dst_QP-only case already defined in the Hash_Input construction above
            (i.e., the 56-bit input).
            This consistently applied relabeling ensures that, in both directions, the concurrent RDMA
            sessions sharing a given 5-tuple are spread across the parallel DCI links, avoiding the
            reverse path collapsing onto a single link.
          </t>
          <t>
            IP fragmentation: the QP information is present only in the first fragment of a fragmented RoCEv2
            packet; subsequent fragments repeat only the IPv6 header and fragment data. Since the ingress
            network device cannot extract the QP numbers from non-first fragments, it SHOULD leave their Flow
            Label unchanged (zero), and such fragments are load-balanced on the remaining IPv6 header fields
            alone, following the same fallback that <xref target="RFC6437"/> Section 3 allows for forwarding
            nodes that cannot access the complete 5-tuple. IPv6 reassembly does not require fragments
            of the same packet to arrive on the same path (the receiver reassembles them by fragment
            offset), so this fallback preserves correctness; fragmented traffic simply does not
            benefit from the QP-derived entropy. In practice, RoCEv2 deployments size RDMA transfers
            to the path MTU, so IP fragmentation of RoCEv2 traffic is rare.
          </t>
          <t>
            IPv6 extension headers: when extension headers are present between the IPv6 header and the UDP
            header, the IB transport headers are no longer at a fixed offset and parsing becomes more
            expensive. An implementation MAY skip relabeling for such packets and leave the Flow Label
            unchanged. In the deployments envisioned here, RoCEv2 packets do not normally carry extension
            headers.
          </t>
        </section>
      </section>
    
    <section anchor="IANA">
    <!-- All drafts are required to have an IANA considerations section. See RFC 8126 for a guide.-->
      <name>IANA Considerations</name>
      <t>This document makes no request to IANA.</t>
    </section>
    
    <section anchor="Security">
      <!-- All drafts are required to have a security considerations section. See RFC 3552 for a guide. -->
      <name>Security Considerations</name>
      <section>
        <name>Security issue</name>
        <t>
          This scheme only modifies the Flow Label field of the IPv6 header; this modification is performed by the
          ingress network device. It does not involve altering the packet payload. As discussed in Section 6.3 of
          <xref target="RFC6437"/>, the Flow Label is not included in the cryptographic calculations of IPsec,
          so relabeling has no effect on IPsec end-to-end security and cannot cause an IPsec integrity check to
          fail. The modification does not change IP addresses or port numbers, thus imposing no additional
          processing burden on existing stateful firewalls or NAT devices.
        </t>
        <t>
          However, as noted in <xref target="RFC6437"/>, the Flow Label is an unprotected field that could be modified en route. Since
          the ingress network device is a trusted infrastructure element within the network operator's domain, the risk
          of malicious Flow Label modification is confined to the same trust boundary as other network devices.
        </t>
        <t>
          One deliberate departure from <xref target="RFC6437"/> deserves explicit mention. Section 6 of
          <xref target="RFC6437"/> states that, when a hash algorithm is used to set the Flow Label, it SHOULD
          include a step that makes the label value significantly difficult to predict. The hash defined in this
          document is purely deterministic over the QP numbers and address bits, all of which are visible to,
          or chosen by, the endpoints. A malicious host could therefore choose QP numbers whose labels collide
          or concentrate onto a particular DCI link in an attempt to create localized congestion. In the
          deployment envisioned here - managed data centers interconnected by operator-controlled DCI links -
          the hosts are administratively managed, so this attack surface is comparable to that created by the existing ability
          of a host to steer conventional ECMP by choosing its source ports. Operators who consider this risk
          material MAY mix a per-device secret value into the hash input, as suggested for a similar case in
          Section 5 of <xref target="RFC7098"/>; in that case, edge devices that may relabel packets of the
          same session MUST share the secret so that the labeling remains consistent.
        </t>
        <t>
          Furthermore, as noted in <xref target="RFC6437"/>, inspection of unencrypted Flow Labels may allow some
          forms of traffic analysis by revealing part of the underlying communication structure. In this method,
          an on-path observer can use the number of distinct labels visible between a given pair of addresses to
          infer a lower bound on the number of concurrent RDMA sessions, although the labels themselves do not
          reveal the actual QP numbers.
        </t>
      </section>
      <section>
        <name>Compatibility issue</name>
        <t>
          End-to-End Protocol: The receiving device typically ignores the Flow Label field, making the scheme completely
          transparent to terminals that support standard IPv6.
        </t>
        <t>
          Intermediate Devices: The scheme delivers its benefit when the ECMP hash of the DCI transit nodes includes
          the Flow Label, which is the direction encouraged by <xref target="RFC6437"/> for stateless load distribution
          across ECMP and Link Aggregation Group (LAG) paths. All network devices supporting the IPv6 Flow Label field can benefit from this
          scheme. For legacy devices whose hash covers only the traditional 5-tuple, packets are still forwarded
          normally: the scheme will not cause connectivity issues, but those devices will not contribute additional
          spreading.
        </t>
        <t>
          Hardware-Friendly Implementation: The CRC32 generator polynomial (0x04C11DB7) is a common hardware
          primitive. The CRC-32 variant used here is the standard reflected formulation that is also used by
          Ethernet and by widely deployed software libraries, so it can be realized with well-understood CRC32
          building blocks in both hardware and software. Implementing the required logic (parsing the BTH and,
          when present, the DETH; performing the hash; and modifying the Flow Label) is relatively
          straightforward and requires minimal changes to existing hardware.
        </t>
      </section>
    </section>
    
    <!-- NOTE: The Acknowledgements section is at the end of this template -->
  </middle>

  <back>
    <references>
      <name>References</name>
      <references>
        <name>Normative References</name>

        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.2119.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.8174.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6437.xml"/>
        <!-- The recommended and simplest way to include a well known reference -->
        
      </references>
      <references>
        <name>Informative References</name>

        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.6438.xml"/>
        <xi:include href="https://bib.ietf.org/public/rfc/bibxml/reference.RFC.7098.xml"/>
      </references>
    </references>

    <section anchor="acknowledgments" numbered="false" toc="default">
      <name>Acknowledgments</name>
      <t>
        The authors would like to thank Brian E Carpenter and Dale W. Carder for their valuable comments and discussion.
      </t>
    </section>
    
 </back>
</rfc>
