IPv6 Hop-by-Hop Options Processing Procedures

This document specifies procedures for processing IPv6 Hop-by-Hop options in IPv6 routers and hosts. It modifies the procedures specified in the IPv6 Protocol Specification to make processing of IPv6 Hop-by-Hop Options header practical with the goal of making IPv6 Hop-by-Hop options useful to deploy and use at IPv6 routers and hosts. The current list of defined Hop-by-Hop options can be found at . The focus for this document is to set a lower bound for the minimum number of Hop-by-Hop options that ought to be processed. This document does not discuss an upper bound. That topic is discussed in . When published, this document updates .

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

This document uses the following loosely defined terms:

Forwarding Plane: IPv6 routers exchange user data through the forwarding plane. Routers process fields contained in packet headers. However, they do not process information contained in packet payloads.
Control Plane: IPv6 routers exchange management and routing information. They also exchange routing information with one another. Management and routing information are processed by its recipient. Management and control information can be forwarded by a router that process fields contained in packet headers.
Fast Path: A path through a router that is optimized for forwarding packets. The Fast Path might be supported by Application Specific Integrated Circuits (ASICS), Network Processor (NP), or other special purpose hardware. This is the usual processing path within a router taken by the forwarding plane.
Slow Path: A path through a router that is capable of general purpose processing and is not optimized for any particular function. This processing path is used for packets that require special processing or differ from assumptions made in Fast Path heuristics or to process router control protocols used by the control plane.
Full Forwarding Rate: This is the rate that a router can forward packets without adversely impacting the aggregate forwarding rate. For example, a router could process packets with Hop-by-Hop options at a rate that allows it to maintain the full speed on its outgoing interfaces, which is sometimes called "wire speed".

NOTE: is an example of how designs can separate control plane and forwarding plane functions. The separation between hardware and software processing described in does not apply to all router architectures. However, a router that performs all or most processing in software might still incur more processing cost when providing special processing for Hop-by-Hop options.

In the first versions of the IPv6 specification and , Hop-by-Hop options were required to be processed by all nodes: routers and hosts. This proved to not be practical in current high speed routers, as observed in Section 2.2 of RFC7045: "it is to be expected that high-performance routers will either ignore it or assign packets containing it to a slow processing path". The reason behind this includes:

Inability to process Hop-by-Hop options at the full forwarding rate can result in issues. In some cases, Hop-by-Hop options would be sent to the Control Plane. This could degrade a router's performance and also its ability to process critical control traffic. Both of which could be exploited as a Denial of Service attack against the router.
If a subset of packets in a flow were to include Hop-by-Hop options, this could introduce a potential for packets to be delivered out of order to the destination. This might result when the Extension Header was included in only some packets, or if a specific Hop-by-Hop option required different processing for some packets in a flow. Significant reordering of the packets belonging to a flow can impact the performance of upper layer protocols and needs to be avoided.
Packets could include multiple Hop-by-Hop options. Too many options could make the previous issues worse by increasing the resources required to process them. The total size of the options determines the number of header bytes that might need to be processed.

specified a uniform format for new IPv6 Extension Headers. It updated and this update was incorporated into Section 4.8 of . When the IPv6 Specification was updated and published in July 2017 as , the procedures relating to Hop-by-Hop options were specified (, Section 4) as follows:

The Hop-by-Hop Options header is not inserted or deleted, but may be examined or processed by any node along a packet's delivery path, until the packet reaches the node (or each of the set of nodes, in the case of multicast) identified in the Destination Address field of the IPv6 header. The Hop-by-Hop Options header, when present, must immediately follow the IPv6 header. Its presence is indicated by the value zero in the Next Header field of the IPv6 header.
NOTE: While required that all nodes must examine and process the Hop-by-Hop Options header, it is now expected that nodes along a packet's delivery path only examine and process the Hop-by-Hop Options header if explicitly configured to do so.

The changes meant that an implementation complied with the IPv6 specification even if it did not process Hop-by-Hop options, and that it was expected that routers would add configuration information to control which Hop-by-Hop options they would process. The text regarding processing of Hop-by-Hop options in was not intended to change the processing of these options. It documented how they were being used in the Internet at the time RFC8200 was published (see Appendix B of ). This was a constraint on publishing the IPv6 specification as an IETF Standard. The main issues remain:

Routers can be configured to drop transit packets containing Hop-by-Hop Options that would have required processing in the Control Plane. This could protect against a denial of service attack on the router .
IPv6 Packets that include a Hop-by-Hop Options header are dropped by some Internet paths. A survey in 2015 reported a high loss rate in transit ASs for packets with Hop-by-Hop options . The operational implications of IPv6 Packets that include Extension Headers are discussed in .
Allowing multiple Hop-by-Hop options in a single packet in some cases consume more router resources to process these packets. It also adds complexity to the number of permutations that might need to be processed/configured.
Including larger or multiple Hop-by-Hop options in a Hop-by-Hop Options header increases the number of bytes that need to be processed in forwarding, which can in some designs impact the cost of processing a packet, and in turn could increase the probability of drop . A larger Extension Header could also reduce the probability that a router can locate all the header bytes required to successfully process an access control list operating on fields after the Hop-by-Hop Options header.
Any option that can be used to force packets into the router's Control Plane can be exploited as a denial of service attack on a transit router by saturating the resources needed for router management protocols (e.g., routing protocols, network management protocols, etc.) that could cause adversely impact router operation. This is an issue for the Router Alert option, which intentionally forward packets to the Control Plane, and is discussed in . This impact could be mitigated by limiting the use of control plane resources by a specific packet, and/or by the use of per-function rate-limiters for packets processed by the control plane.

Section 3 of RFC 6398 includes a summary of processing the IP Router Alert Option:

"In a nutshell, the IP Router Alert Option does not provide a convenient universal mechanism to accurately and reliably distinguish between IP Router Alert packets of interest and unwanted IP Router Alert packets. This, in turn, creates a security concern when the IP Router Alert Option is used, because, short of appropriate router-implementation-specific mechanisms, the router Slow Path is at risk of being flooded by unwanted traffic."

This is an example of the need to limit the resources that can be consumed when a particular function is executed and to avoid consuming control-plane resources where support for a function has not been configured. There has been research that has discussed the general problem with dropping packets containing IPv6 Extension Headers, including the Hop-by-Hop Options header. For example, states that "dropping all packets with Extension Headers, is a bad practice", and that "The share of traffic containing more than one EH however, is very small. For the design of hardware able to handle the dynamic nature of Extension Headers we therefore recommend to support at least one EH". Operational aspects of the topics discussed in this section are further discussed in . "Transmission and Processing of IPv6 Extension Headers" clarified how intermediate nodes should process Extension Headers. This document is generally consistent with , and was considered when was updated and was itself replaced by . This document updates as described in the next section and consequently clarifies the description in Section 2.2 of , using the language of BCP 14 . The document defines a set of procedures for the Hop-by-Hop Options header that are intended to make the processing of Hop-by-Hop options practical in modern transit routers. The authors expectations are that some Hop-by-Hop options will be processed across the Internet, while others will only be processed within a limited domain (e.g., where a specific service is made available in that network segment that relies on one or more Hop-by-Hop options).

This section describes several changes to . describes processing of the Hop-by-Hop option Extension Header, and describes processing of individual Hop-by-Hop Options.

When a packet includes one or more Extension Headers, the Next Header field of the IPv6 Header does not identify the transport protocol. The Extension Header used to carry Hop-by-Hop options is defined in Section 4.3 of and is identified by a Next Header value of 0 in the IPv6 header. Section 4.1 of requires this Hop-by-Hop Options header to appear immediately after the IPv6 header. also requires that a Hop-by-Hop Options header can only appear once in a packet. The Hop-by-Hop Options Header as defined in can contain one or more Hop-by-Hop options. This document updates : Routers SHOULD process the Hop-by-Hop Options header using the method defined in this document. If a router does not process the Hop-by-Hop Options header, it MUST forward the packet normally based on the remaining Extension Header after the Hop-by-Hop option (i.e., a router must not drop a packet solely because it contains an Extension Header carrying Hop-by-Hop options). A configuration could control that normal processing skips any or all of the Hop-by-Hop options carried in the Hop-by-Hop Options header. It is expected that the Hop-by-Hop Options header will be processed by the Destination. Hosts SHOULD process the Hop-by-Hop Options header in received packets. If a Destination does not process the Hop-by-Hop Options header, it MUST process the remainder of the packet normally. Further details on requirements for host processing are described in .

Section 4 of allows a router to control its processing of IPv6 Hop-by-Hop options by local configuration. The text is:

NOTE: While required that all nodes must examine and process the Hop-by-Hop Options header, it is now expected that nodes along the path only examine and process the Hop-by-Hop Options header if explicitly configured to do so.

A configuration could control whether processing skips any specific Hop-by-Hop options carried in the Hop-by-Hop Options header. A router that does not process the contents of the Hop-by-Hop Options header does not therefore process the identifiers of individual Option Types to perform any specified action.

A Source creating packets with a Hop-by-Hop Options header SHOULD use a method that is robust to network nodes processing only the first Hop-by-Hop option that is included in the packet, or that forward packets without the option being processed (see ). A Source MAY, based on local configuration, include more than one Hop-by-Hop option , but might wish to restrict the size to increase the likelihood of successful transfer across a network path. This motivates Sources to order the placement of Hop-by-Hop options within the Hop-by-Hop Options header in decreasing order of importance for their processing by nodes on the path. A router MUST NOT be configured to support the Hop-by-Hop Option header if it cannot process the first Hop-by-Hop option at full forwarding rate. If configured to do so, a router SHOULD process additional Hop-by-Hop options providing that these also can be processed at the full forwarding rate. If a router is unable to process any Hop-by-Hop option (or is not configured to do so), it SHOULD behave in the way specified for an unrecognized Option Type when the action bits were set to "00". If a router is unable to process further Hop-by-Hop options (or is not configured to do so), the router SHOULD skip the remaining options using the "Hdr Ext Len" field in the Hop-by-Hop Options header. This field specifies the length of the Option Header in 8-octet units. After skipping an option, the router continues processing the remaining options in the header. Skipped options do not need to be verified. The Router Alert option is an exception that can result in processing in the Control Plane, see . Section 4.2 of defines the Option Type identifiers as internally encoded such that their highest-order 2 bits specify the action that must be taken if the processing IPv6 node does not recognize the Option Type. The text is: This document modifies this behaviour for the "01" value to relax the requirement to drop a packet. It also modifies the behaviour for the "10" and "11" values that the node MAY send an ICMP Parameter Problem, Code 2, message to the packet's Source Address, pointing to the unrecognized Option Type. The modified text for "01","10" and "11" values is: When a node sends an ICMP message in response to a multicast packet, this could be exploited as an amplification attack. This is particularly problematic when the source address is not valid (which can be mitigated when using a reverse path forwarding (RPF) check). A node SHOULD only send ICMP messages in response to a multicast address when this is enabled for the specific source and/or the group destination address. When an ICMP Parameter Problem, Code 2, message is delivered to the source, the source can become aware that at least one node on the path has failed to recognize the option. Generating an ICMP message incurs additional router processing. Reception of this message is not guaranteed, routers might be unable to be configured so that they do not generate these messages, and they are not always forwarded to the Source. The motivation here is to loosen the requirement to send an ICMPv6 Parameter Problem message when a router forwards a packet without processing the list of all options.

The purpose of the Router Alert option is to tell a router that the packet needs additional processing in the Control Plane. The Router Alert option includes a two-octet Value field that describes the protocol that is carried in the packet. The current specified values can be found in the IANA Router Alert Value registry . DISCUSSION

The function of a Router Alert option results in the processing that this specification is proposing to eliminate, that is, to instruct a router to process the packet in the Control Plane. The function of a Router Alert option is an instruction to a router to process the packet in the Control Plane that results in the concerns discussed in section 4. One approach would be to deprecate this, because current usage beyond the local network appears to be limited, and packets containing Hop-by-Hop options are frequently dropped. Deprecation would allow current implementations to continue and its use could be phased out over time.
The Router Alert option could have a potential for use with new functions that have to be processed in the Control Plane. Keeping this as the single exception for processing in the Control Plane with the following restrictions is a reasonable compromise to allow future flexibility. These are compatible with Section 5 of .

Implementations of the IP Router Alert option SHOULD offer the configuration option to simply ignore the presence of "IP Router Alert" in IPv4 and IPv6 packets" . A node that is configured to process a Router Alert option MUST protect itself from infrastructure attack that could result from processing in the Control Plane. This might include some combination of an access control list to only permit this from trusted nodes, rate limiting of processing, or other methods . As specified in the top two bits of Option Type for the Router Alert option are always set to "00" indicating the node should skip over this option and continue processing the header. An implementation SHOULD verify that a Router Alert option contains a protocol, as indicated by the Value field in the Router Alert option, that is configured as a protocol of interest to that router. A verified packet SHOULD be sent to the Control Plane for further processing . Otherwise, the router implementation SHOULD forward this packet subject to all normal policies and forwarding rules).

A router can be configured to process a specific Option. A possible approach to implementing this is to maintain a lookup table based on Option Type of the IPv6 options that can be processed at full forwarding rate. This would allow a router to quickly determine if an option is supported and can be processed. If the option is not supported, then the router processes the option as described in of this document. The actions of the lookup table SHOULD be configurable by the operator of the router.

This section updates Section 4.8 of . Any new IPv6 Hop-by-Hop option designed in the future should be designed to be processed at full forwarding rate (e.g., without slowing processing of other packets). New Hop-by-Hop options should have the following characteristics:

New Hop-by-Hop options SHOULD be designed to ensure the router can process the options at the full forwarding rate. That is, they should be simple to process, see .
New options SHOULD be defined with the Action type (highest-order 2 bits of the Option Type) set to 00 to skip over this option and continue processing the header if a router does not recognize the option.
The size of a Hop-by-Hop option SHOULD NOT extend beyond what can be expected to be executed at full forwarding rate. A larger Hop-by-Hop Option header can increase the likelihood that that a packet will be dropped.
New Hop-by-Hop options SHOULD be designed expecting that a router might be configured to only process a subset of Hop-by-Hop options (e.g., the first) in the Hop-by-Hop Options header.
New Hop-by-Hop options SHOULD be designed expecting that a router may drop packets containing the new Hop-by-Hop option.

Any new Hop-by-Hop option that is standardized that does not meet these criteria MUST include in the specification a detailed explanation why this can not be accomplished and to show that there is a reasonable expectation that the option can be proceed at full forwarding rate. This is consistent with . The general issue of robust operation of packets with new Hop-by-Hop options is described in below.

Recent measurement surveys (e.g., ) show that packets that include Extension Headers can cause the packets to be dropped by some Internet paths. In a controlled domain, routers can be configured or updated to provide support for any required Hop-by-Hop options. The primary motivation of this document is to make it more practicable to use Hop-by-Hop options beyond such a single domain, with the expectation that applications can improve the quality of or add new features to their offered service when the path successfully forwards packets with the required Hop-by-Hop options(s), and otherwise refrains from using these option(s). The focus is on incremental deployability. A protocol feature (such as using Hop-by-Hop options) is incrementally deployable if early adopters gain some benefit over the paths being used, even though other paths do not support the protocol feature. A Source ought to order the Hop-by-Hop options that are carried in the Hop-by-Hop Options header in decreasing order of importance for processing by nodes on the path. Methods can be developed that do not rely upon all routers to implement a specific Hop-by-Hop option (e.g., , and that are robust when the current path drops packets that contain a Hop-by-Hop option (e.g., ). For example, an application can be designed to first send a test packet(s) that includes the required option, or combination of options, and sends other packets without including the option. The application then does not send additional packets that include this option (or set of options) until the test packet(s) is acknowledged. The need for potential loss recovery when a path drops these test packets can be avoided by choosing packets that do not carry application data that needs to be reliably delivered. Since the set of nodes forming a path can change with time, this discovery process ought to be repeated from time-to-time. The process of sending packets both with and without a specific header to discover whether a path can support a specific header is sometimes called "racing" (e.g., transport protocol racing is explained in , or "A/B protocol feature testing" is described in ).

There are no actions required for IANA defined in this document.

Security issues with including IPv6 Hop-by-Hop options are well known and have been documented in several places, including , , and . The main issue, as noted in , is that any mechanism that can be used to force packets into the router's Control Plane can be exploited as a denial of service attack on a transit router by saturating the resources needed for router management (e.g., routing protocols, network management protocols, etc.) that may cause the router to fail or perform sub-optimally. This is because it has been observed that certain routers drop packets that include a Hop-by-Hop Options header. While Hop-by-Hop options are not required to be processed in the Control Plane, the Router Alert option is designed to do just that. Some IPv6 nodes implement features that access more of the protocol information than a typical IPv6 router (e.g., ). Examples are nodes that provide DDOS mitigation, Firewall/access control, traffic engineering, or traffic normalization. These nodes could be configured to drop packets when they are unable to access and process all Extension Headers, or are unable to locate and process the higher-layer packet information. This document provides guidance on the requirements concerning Hop-by-Hop options. Finally, the document notes that Internet protocol processing needs to be robust to malformed/malicious protocol fields. This requirement is not specific to Hop-by-Hop options. It is important that implementations fail gracefully when a malformed or malicious Hop-by-Hop option is encountered. This document changes the way the Hop-by-Hop Options header is processed in several ways that significantly reduce the attack surface. These changes include:

All Hop-by-Hop options (with one exception) must be processed at full forwarding rate. Only one Hop-by-Hop option MUST be processed and additional Hop-by-Hop options MAY be processed based on local configuration.
It adds criteria for the Router Alert option to allow control over how the Router Alert option is processed and that a node configured to support these options must protect itself from attacks using the Router Alert option.
The document limited the default number of Hop-by-Hop options that can be included in a packet to a single Hop-by-Hop option.
Additional Hop-by-Hop options MAY be included, based on local configuration. Although nodes only process these additional Hop-by-Hop options if configured to do so.
The document added restrictions to any future new Hop-by-Hop option that limit their size and computational requirements.

The authors intent is that these changes significantly reduce the security issues relating to processing the IPv6 Hop-by-Hop Options header and to enable Hop-by-Hop options to be safely used in the Internet.

Helpful comments were received from Brian Carpenter, Ron Bonica, Ole Troan, Mark Heard, Tom Herbert, Cheng Li, Eric Vyncke, Greg Mirksy, Xiao Min, Fernando Gont, Darren Dukes, Peng Shuping, Dave Thaler, Ana Custura, Tim Winters, Fernando Gont, Jingrong Xie, Lorenzo Colitti, Toerless Eckert, Suresh Krishnan, Mikael Abrahamsson, and other members of the 6MAN working group.

draft-ietf-6man-hbh-processing-10, 2023-September-26:

Clarifying changes based on comments received during the IPv6 w.g. session at IETF117 from Lorenzo Colitti, Toerless Eckert, and others.
Clarifying changes based on comments received after the first w.g. last call.
Editorial Changes.

draft-ietf-6man-hbh-processing-09, 2023-July-4:

Revised text in relating to fast/slow path and processing
Restructured to separate Hop-by-Hop Options header and Hop-by-Hop options processing and configuration.
Revised MUST/SHOULD language in .
Revised text to use consistant names for Hop-by-Hop Options header and Hop-by-Hop options.
Revised regarding the modified behaviour of the action bits "01", "02", and 03" to be a MAY to be consistant with text earlier in that section.
Added to that new Hop-by-Hop options SHOULD be designed expecting that routers may drop packets with the new option.
Added new on "Example of Robust Usage".
Other editorial changes to improve readability and clarity.

draft-ietf-6man-hbh-processing-08, 2023-April-30:

Changed document that is no longer updates , it now clarifies it using the language of BCP 14.
Added additional clarification to .
Editorial changes

draft-ietf-6man-hbh-processing-07, 2023-April-6:

Changed text to clarify how hosts and routers process the Hop-by-Hop Options header based on comments at 6MAN session at IETF 116.
Editorial changes

draft-ietf-6man-hbh-processing-06, 2023-March-11:

Added reference to RFC6564.
Editorial changes

draft-ietf-6man-hbh-processing-05, 2023-February-23:

Clarified text in about processing complexity and time to process.
Added a definition to for "Full Forwarding Rate".
Added text to about nodes that do not process the Hop-by-Hop Options header.
Added text to about slow path processing can cause packets to be deliver out of order to the destination.
Editorial changes

draft-ietf-6man-hbh-processing-04, 2022-October-21:

Add a paragraph to that describes the relationship to "Transmission and Processing of IPv6 Extension Headers".
Change that this draft updates section 2.2 of .

draft-ietf-6man-hbh-processing-03, 2022-October-12:

Changed in to have router skip over options if can't process at full forwarding rate.
Added to that new options should be defined with action type set to 00.

draft-ietf-6man-hbh-processing-02, 2022-August-23:

Several clarification and editorial changes suggested by a review by Peng Shuping.
Editorial changes.
Revised text relating to fast/slow path and processing rates.
Revised the third paragraph in to be clearer.
Revised text in Security section based on comments from Fernando Gont.

draft-ietf-6man-hbh-processing-01, 2022-June-15:

Fixed typo in last paragraph of
Revised text in to reflect constraints on publishing RFC8200.
Changed text in that new options SHOULD NOT (from MUST NOT) be defined that require that are not expected to be excepted at full forwarding rates.
Added reference to RFC7872 in .
Added text to that the focus of this document is to set a minimum bound on the number of Hop-by-Hop options a node should process.
Added text to that the authors some Hop-by-Hop options will be supported Internet wide, and others only in limited domains.
Editorial changes.

draft-ietf-6man-hbh-processing-00, 2022-January-29:

6MAN Working Group Draft
Reworked text to talk about processing Hop-by-Hop options at full forwarding rates, instead of "fast path"
Revised "New Hop-by-Hop options" to allow variable sized Hop-by-Hop options, remove specific length requirements, and other clarifications.
Editorial changes.

draft-hinden-6man-hbh-processing-01, 2021-June-2:

Expanded terminology section to include Forwarding Plane and Control Plane.
Changed draft that only one Hop-by-Hop option MUST be processed and additional Hop-by-Hop options MAY be processed based on local configuration.
Clarified that all Hop-by-Hop options (with one exception) must be processed on the Fast Path.
Kept the Router Alert option as the single exception for Slow Path processing.
Rewrote and expanded section on New Hop-by-Hop options.
Removed requirement for Hop-by-Hop option size and alignment.
Removed sections evaluating currently defined Hop-by-Hop options.
Added content to the Security Considerations section.
Added people to the acknowledgements section.
Numerous editorial changes

draft-hinden-6man-hbh-processing-00, 2020-Nov-29:

Initial draft.