Mobile User Plane Evolution

Current User Plane in 5G Mobile User Plane (MUP) in 5G has two distinct parts: the Access Network part between UE and AN/gNB, and the Core Network part between AN/gNB and UPF. For the core network (CN) part, N3 interface extends the PDU layer from AN/gNB towards the PSA UPF, optionally through I-UPFs and in that case N9 interface is used between I-UPF and PSA UPF. Traditionally, UPFs are deployed at central locations and the N3/N9 tunnels extend the PDU layer to them. The N3/N9 interface uses GTP-U tunnels that are typically over a VPN over a transport infrastructure. While N6 is a 3GPP defined interface, it is for reference only and there is no tunneling or specification involved. It is simply a direct IP (in case of IP PDU session) or Ethernet (in case of Ethernet PDU session) connection to the DN. At the AN/gNB, relay is done between the radio layer and the GTP-U layer. At the PSA UPF, routing/switching is done for IP/Ethernet before GTP-U encapsulation (for downlink traffic) or after GTP-U decapsulation (for uplink traffic).

MUP Evolution in 5G: Distributed UPFs With MEC, ULCL UPFs are deployed closer to gNBs, while centralized PSA UPFs are still used to provide persistent IP addresses to UEs. In fact, even PSA UPFs could be distributed closer to gNBs and then the N3 interface becomes very simple – over a direct or short transport connection between gNB and UPF (or even an internal connection if the gNB and UPF are hosted on the same server). On the other hand, since the UPF to DN connection is direct, the DN becomes a VPN (e.g., IP VPN in case of IP PDU sessions or EVPN in case of Ethernet PDU sessions) over a transport infrastructure, most likely the same transport infrastructure for the VPN supporting the N3/N9 tunneling in centralized PSA UPF case, as shown in the following picture: The central PSA UPF is no longer needed in this case. Distributed UPF1/UPF2 connect to VRF1 on PE1/PE2 and VRF1 is for the VPN of the DN that UE1/UE2 access. There is also a PE3 for other sites of the VPN, which could be wireline sites including sites providing Internet access. UEs may keep their persistent IP addresses even when they re-anchor from one PSA UPF to another. In that case, for downlink traffic to be sent to the right UPF, when a UE anchors at a UPF the UPF advertises a host route for the UE and when a UE de-achors from a UPF the UPF withdraws the host route. While this relies on host routes to direct to-UE traffic to the right UPF, it does not introduce additional scaling burden compared to centralized PSA UPF model, as the centralized UPFs need to maintain per-UE forwarding state (in the form of PDRs/FARs) and the number is the same as the number of host routes that a hub DN router (e.g. vrf1 on PE3 for internet access) need to maintain in the distributed PSA UPFs model. Since the host routes may be lighter-weighted than the PDRs/FARs, the total amount of state may be actually smaller in the distributed model. For UE-UE traffic, the distributed PSA UPFs may maintain host routes that they learn from each other. With that the UE-UE traffic may take direct UPF-UPF path instead of going through a hub router in the DN (equivalent of central UPF). That is important in LAN-type services that require low delay. Alternatively, the distributed UPFs may maintain only a default route pointing to the hub router like PE3 (besides the host routes for locally anchored UEs). That way, they don't need to maintain many host routes though UPF-UPF traffic has to go through the hub router (and that is similar to all traffic going through a central PSA UPF). Optionally, even the host routes for locally anchored UEs can be omitted in the FIB of local UPF. Traffic among local UEs can be simply routed to the hub router following the default route, who will then send back to local UPF using VPN tunnels (MPLS or SRv6) that are stitched to GTP tunnels for destination UEs.

Advantages of Distributed PSA UPFs Distributed PSA UPFs have the following advantages:

MEC becomes much simpler - no need for centralized PSA UPF plus ULCL UPFs, and no need for special procedures for location based edge server discovery.
For LAN-type services, UE-UE traffic can be optimized (no need to go through centralized PSA UPFs) when UPFs maintain host routes. It also allows seamless integration of services across wireline/wireless-connected customer sites.
N3/N9 tunneling is simplified

In particular, there is now only short/simple N3 tunneling between AN/gNB and local UPFs in proximity. Among the distributed UPFs and other DN sites, versatile IETF/wireline VPN technologies are used instead. For example:

Any tunneling technology - MPLS, SR-MPLS or SRV6 - with any traffic engineering/differentiation capabilities can be used. Removal of the GTP/UDP header (and IPv4/IPv6 header in case of MPLS data plane) brings additional bandwidth savings in the transport infrastructure.
Any control plane model for VPN can be used - traditional distributed or newer controller based route advertisement.

In short, the distributed PSA UPFs model achieves "N3/N9/N6 shortcut and central UPF bypass", which is desired by many operators. Notice that, since UPF has routing functions, depending on the capability of a UPF device, it may even be possible for a UPF device to act as a VPN PE. That can be done in one of the two models:

The UPF function and VPN PE function are separate but co-hosted on the same device with a logical/internal N6 connection between them.
The UPF and VPN PE function are integrated and the PDU sessions become VPN PE-CE links.

The second model is especially useful when a UE is multi-homed to different EVPN PEs in case of Ethernet PDU sessions - EVPN's all-active multihoming procedures can be utilized.

Extent of Distribution and O-RAN The UPFs can be distributed as close to the gNB as being co-located with it - either with a direct local link in between or both running as virtual functions on the same compute server. In the O-RAN architecture , the gNB function is split into gNB-CU (O-RAN CU or O-CU, for Central Unit) and gNB-DU (O-RAN DU or O-DU, for Distributed Unit). O-CU is the N3 GTP tunnel endpoint and is what gNB refers to in this document. Thus, the centralization process of the O-CU component can converge with the distribution process of the UPF up to some optimal and convenient location in the network.

Enablers of Distributed PSA UPFs To distribute PSA UPFs, if persistent addresses must be used for UEs, the SMF must be able to allocate persistent IP addresses from a central pool even when a UE re-anchors at different PSA UPFs (e.g. due to mobility). If DHCPv4 is used, either the SMF acts as a central DHCP server or it relays DCHP requests to a central DHCP server on the DN. The distributed PSA UPFs must be able to advertise host routes in the DN. This should not be a problem since a UPF is essentially a router in that it routes traffic between DN and UEs (that are connected via PDU sessions). Notice that, advertising host routes for persistent IP addresses is no different from advertising MAC addresses in case of Ethernet PDU sessions.

MUP Evolution in 5G: Alternative Implementation Options

GTP vs. SRv6 vs. MPLS tunneling 3GPP specifies that all tunneling (e.g. N3/N9) use GTP, whose encapsulation includes IP header, UDP header and GTP header. The tunnel is between 3GPP NFs (e.g. gNBs and UPFs) over an IP transport, and the IP transport may be a VPN over the multi-service transport infrastructure of an operator. There have been proposals to replace GTP with SRv6 tunnels for the following benefits:

Traffic Engineering (TE) and Service Function Chaining (SFC) capability provided by SRv6
Bandwidth savings because UDP and GTP headers are no longer needed

While 3GPP has not adopted the proposal, and GTP can be transported over SRv6 (as overlay, instead of SRv6 replacing GTP), some operators still prefer to replace GTP with SRv6 "under the hood". That is, while RAN/UPF still use N2/N4 signaling, the actual tunnel are no longer GTP but SRv6 based on GTP parameters signaled by N2/N4. The SRv6 tunnel could be between two NFs, or a GW could be attached to an NF that still use traditional GTP and the GW will convert GTP to/from SRv6. This is specified in . Similarly, if an operator prefers to use MPLS, a GTP tunnel can also be replaced with an MPLS PW instead of an SRv6 tunnel. Compared with SRv6, it is even more bandwidth efficient (no need for a minimum 40-byte IPv6 header) and SR-MPLS can also provide TE/SFC capabilities. This is specified in . Note that, While only IPv6 can scale to the 5G requirements for the transport infrastructure, it does not mean MPLS can not be used as data plane in the IPv6 network.

Routing Based UPF-Lite Traditionally, a UPF is implemented to follow 3GPP specifications. Specifically, N4 signaling is used for SMF to instruct a UPF to set up its session state in terms of PDRs/FARs. On N6 side, a UPF receives downlink traffic with destination addresses that are covered by the UPF's address range for its anchored UEs. The packet is matched against the installed PDRs and forwarded according to the associated FARs. On N3 side, a UPF decapsulates GTP+UDP+IP header of uplink traffic and uses the TEID to identify the DN where inner IP routing or Ethernet switching is done. specifies a new SRv6 based MUP architecture. When it is applied to a 3GPP based mobile architecture:

BGP signaling from a MUP Controller replaces N4 signaling from SMF. N4 signaling is still used between the MUP Controller and SMF - from SMF's point of view it is just interacting with a traditional UPF as usual.
A MUP GW becomes a distributed UPF for uplink traffic.
A MUP PE, which is different from a usually central PSA UPF, becomes a UPF for downlink traffic, in that traffic to each UE is placed into a different tunnel that is stitched to a GTP tunnel for that UE by a MUP GW (no route lookup is needed on the MUP GW for the downlink traffic).

In this approach UE to UE traffic may still optionally go through the central PSA UPF. This is similar to that a hub router may be used in . This approach can be viewed as a specific way of implementing/deploying a subset of functionalities of distributed UPFs discussed in , specifically the routing/switching functionalities, hence often referred to as UPF-Lite. It does have the advantage that from SMF's point of view, nothing is different from before - both from N4 signaling and deployment model point of view. While the above is specific to SRv6, a similar MPLS based approach will be specified separately for operators who prefer MPLS data plane, and it can even be SR-agnostic.

MUP Evolution in 6G This section discusses potential MUP evolution in 6G mobile networks. It does involve changes in 3GPP architecture and signaling, so the purpose is to share the ideas in IETF/wireline community first. If it gains consensus within IETF/wireline community especially among mobile operators, then the proposal may be brought to 3GPP community for further discussions.

UPF Distribution and RAN Decomposition As described earlier, with 5G, in the opposite direction of UPF distribution, some RAN components are becoming centralized as a result of the disaggregation and decomposition of baseband processing functions. The AN functionality is now divided into the Radio Unit (RU, comprising the antenna and radiating elements), the Distributed Unit (DU, comprising the functions for the real time processing of the signal), and the Centralized Unit (CU, comprising the remaining signal processing functions). CU is the AN function that handles N3 GTP-U encapsulation for UpLink (UL) traffic and decapsulation for DownLink (DL) traffic. This is also specified in , with corresponding O-RU, O-DU and O-CU terms. The placement of the decomposed CU component can converge with the distribution process of the UPF to some optimal and convenient location in the network - they become co-located in an edge or far edge data center (DC) either with direct/short local connections in between or both running as virtual functions on the same compute server.

Integrated AN/UP Function (ANUP) While the AN (CU) and UPF can be co-located, in 5G they are still separate functions connected by N3 tunneling over a short/internal transport connection. Routing happens on the UPF between the DN and UEs over the N3 tunnels, and relay happens on the AN between the N3 tunnels and AN protocol stack. With AN and UPF functions more and more disaggregated and virtualized even in 5G, it is becoming more and more feasible and attractive to integrate the AN and UPF functions, eliminating the N3 tunneling and the relay on AN entirely. The combined function is referred to as ANUP in this document, which does routing between DN and UEs over the AN protocol stack directly: With this architecture, 3GPP and IETF technologies are applied where they are best applicable: 3GPP technologies responsible for radio access and IETF technologies for the rest. As IETF technologies continue to evolve, they can be automatically applied in mobile networks without any changes in 3GPP architecture/specification. One way to view this is that the ANUP is a router/switch with wireless and wired interfaces and it routes/switches traffic among those interfaces. The wireless interface is established by 3GPP technologies (just like an Ethernet interface is established by IEEE technologies) and the routing/switching function follows IETF/IEEE standards. Some advantages of this new architecture include:

5G-LAN and MEC become transparent applications that wireline networks have been supporting (PDU sessions terminate into the closest ANUP and routed/switched to various DNs).
MBS becomes very simple - the ANUP gets the multicast traffic in the DN and then use either shared radio bearer or individual bearers to send to interested UEs.
Simplified signaling - instead of seven-steps of separate N2/N4 signaling from separate AMF/SMF to separate AN/UPF and N11 signaling between AMF and SMF to set up the N3 tunneling for a PDU session, a two-step signaling between a new single control plane entity to the single integrated ANUP is enough - see for details.
Simplified/Optimized data plane - AN-UPF connection and GTP-U encapsulation/decapsulation are not needed anymore. This can significantly improve throughput, especially when compared to AN/UPF functions running on servers.
Natural local break-out in traffic forwarding, by allowing the more efficient routing/switching of traffic according to its destination.
Any kind of tunnels can be used for the DN VPN, whether it is MPLS or SRv6, w/o the overhead of UDP/GTP encapsulation compared to GTP tunneling. Network slicing and QoS functions are still supported (even with current GTP tunneling the transport network need to instantiate slices and implement QoS for N3/N9 tunnels as well).

Because the ANUP already implement the routing/switching functions, even the PE functions (for the DN VPN) could be optionally integrated into it, further streamlining end-to-end communication by reducing NFs and connections between them. While integrating PE function is optional, it is desired and today's AN can be already considered as a PE ().

Existing ANUP-like Integration in 3GPP While the proposed the new concept, i.e., the integrated AN and UPF, or ANUP, for the evolution of 6G MUP, this is actually not a completely brand-new idea in 3GPP.

Local IP Access (LIPA) in 4G The 3GPP specification 23.401 standardizes an ANUP-like function, i.e., the Local IP Access or LIPA, that fundamentally integrates together the 4G RAN entity 'HeNB or Home eNodeB' and the traffic switching gateway 'L-GW or Local Gateway'. | +--+---+\ +-----+ +------+ \----/ UP| | \S1-MME /S11 | |Uu \ / | +-----+ +------+ | | | | MME | +--+ UE | +------+ +-----+ ]]> The above figure shows the LIPA architecture. It enables a UE (on the bottom-left) that can connect via a HeNB to access the DN without the user plane traversing the mobile operator's network (e.g., SGW->P-GW). The LIPA feature is achieved using a L-GW (on the top-left) that is collocated with the HeNB. The functionalities of HeNB and L-GW are integrated together to provide the direct User-Plane (UP) path between the HeNB and the L-GW. Please note that there is NO interface between HeNB and L-GW. That is, they are truly an integrated entity. As of now, while the LIPA feature has not yet been deployed extensively by MNO's, it does give somewhat promising indicator that the ANUP-like integration solution has been studied before by 3GPP and it is worthy of the continuous exploration.

Wireless Access via Satellite Network in 5G The 3GPP SA2 working group has two projects to investigate the 5G services whose wireless access capabilities are provided via satellite networks. One project is the Rel-18 SAT_Ph2 that had enjoyed the completion of the stage-2 work in June 2023. The other is a potential Rel-19 project, i.e., SAT_Ph3, whose themes and objectives are still being debated right now. Thanks to the everlasting movement of LEO-based satellites, the Rel-18 SA2_Ph2 project focuses on the support of wireless access under the satellite-based discontinuous coverage. Further, the potential Rel-19 SAT_Ph3 project studies service requirements via satellite access taking into account 5G new capabilities. Regardless, both projects consider the scenario that a gNB will be on board satellite while the corresponding anchor UPF may (i.e., on-board a satellite) or may not (i.e., on the ground). In order to reduce the signaling impact to the target RAT or 5G system, UEs have to remain with no service or not attempt to re-register during the discontinuous satellite coverage. --+ /Core Network +---+ DNN | +-------+ \ / | (gNB + UPF) | | | \--------/ +----------------+ +--------+ UE/GS via Satellite-based Mobile Access Network ]]> While a UPF is on-board satellite, it might not share the same underlaying satellite with the matching gNB. Given the highly mobile satellite constellation network, this will significantly impact the signaling performance between the gNB and the UPF. Some other features are also been investigated, e.g., UE-to-UE communication via satellite(s) without going through the ground network, UE LAN using satellite access, etc. All of these will have to face more complicated requirements if gNB and UPF are deployed on different satellites. On the other aspect, if we plug into the above picture the integrated ANUP solution, there is no more implication of the distribution of gNB and UPF. The complexity of both the CP signaling exchanges and the UP data transport will be greatly relieved. Given the ubiquitous discussion of the satellite communication for 5G, Beyond-5G and later 6G, we do believe our proposal ANUP will highly likely win attractions from both the IETF and the 3GPP communities.

Some considerations with integrated ANUP Various considerations/concerns were brought up during the discussions of the ANUP proposal. They are documented in the following sections.

Separate AN/UP Functions There are still cases where separate AN/UP functions are desired/required:

An MNO may want to deploy one UPF for a cluster of ANs in proximity in some scenarios/locations
An MNO may support MVNOs who have their own UP functions but make use of the hosting MNO's ANs
Home Routed roaming requires separate HPLMN UPs and VPLMN ANs

Therefore, the integration does not have to be always used. Rather, it is "integration when desired and feasible, separation when necessary". Note that, the same ANUP can handle both situations - some PDU sessions may be tunneled to a separate UPF while other sessions are terminated and then traffic is routed/switched to either local DN or remote/central DN. This is also the basis of interworking between 5G and xG:

A 5G AN can have N3 tunneling to an xG UPF
An xG ANUP can have N3 tunneling to a 5G/xG UPF

Simplified/reduced Signaling and optimized data plane One may ask why bother with integration when it is still needed to support separate AN and UPF anyway. When AN and UPF are separate, to set up the N3 tunnel the following seven steps are needed, involving four NFs and three Nx interfaces:

SMF sends request to UPF (N4)
UPF responds with UPF-TEID (N4)
SMF passes <UPF, UPF-TEID> to AMF (N11)
AMF sends request to gNB, passing <UPF, UPF-TEID> (N2)
gNB responds with AN-TEID (N2)
AMF passes <AN, AN-TEID> to SMF (N11)
SMF sends <AN, AN-TEID> to UPF (N4)

With integrated ANUP, there is no need for N3 tunnel anymore. A new control plane NF only needs to tell the ANUP which DN that PDU session belongs to. Additionally, the N3 tunnel is maintained by periodical signaling refreshes - otherwise timeout will happen. This causes significant control plane load on the NFs and interfaces, which no longer exists with ANUP since N3 tunneling is eliminated. As mentioned before, with ANUP the AN-UPF connection and GTP-U encapsulation/decapsulation are not needed anymore. This can significantly improve performance/throughput, especially when compared to AN/UPF functions running on servers.

Mobility Handover Notice that ANUP is for the scenario of distributed UPFs (that are co-located with ANs) and the handover procedures for distributed UPFs (that are not integrated with ANs) applies to ANUP transparently as well. UEs may have persistent IP addresses even when they re-anchor from one ANUP to another, as described in Section 2 of , or they can just get a new address when they re-anchor to a different ANUP, in which case host routes are not needed.

Paging In a mobile system like 5GS the UE may be in power-saving state when the mobile system receives a downlink packet targeted to the UE. In 5GS the UPF is responsible to buffer the packet and notify the SMF and AMF that a downlink data is pending. AMF then instructs the RAN to page the UE, i.e. broadcast a signal to the UE to wake-up from the power-saving state (RRC-Idle or RRC-Inactive state). After receiving the paging the UE reconnects to the gNB and N3 tunnel can be established between the UPF and gNB to deliver the buffered data to the UE. The UE may also move under a new gNB while in a power-saving state; in this case the UE does not connect to a new gNB until receiving the paging message. With integrated ANUP, the UP in ANUP would receive such downlink data packet while the UE is in power-saving state. If the UE has moved out from this ANUP while in power-saving state, and is camping in another (target) ANUP when the source ANUP receives the downlink data packet, upon paging it reconnects to to the target ANUP and may preserve the IP address as described in section . The source ANUP learns the new route for the UE and forwards the buffered data to the target ANUP. Another option is to re-use the RAN-based Notification Area as specified in 5GS. In this case the ANUP that buffers the data is responsible to page the UE across all ANUPs within the RAN-based Notification Area, using the XnAP protocol over the Xn-C interface between the ANUPs. If the UE wakes-up in a new target ANUP the UE could re-anchor to the target ANUP as described above. Again, notice that because ANUP is just the integration of previously separate but co-located AN and UPF functions, the above paging procedures are not different from when AN and UPF are separate.

Microservice architecture One may argue that the integration of AN and UP functions are against the microservice trend. The following is a verbatim quote from https://microservices.io/: The counter argument is that microservice is about decomposing complex "applications". ANUP is about integrating co-located and mature data plane entities to streamline and optimize forwarding. It has real and significant benefits of simplified signaling and optimized data plane - it does not make sense to force microservice here for data plane. Note that microservices can still be utilized in the control plane for ANUP.

Increased burden on previously simple AN One may think that the AN only needed to do simple traffic stitching functions while now the ANUP has added UPF burden. However, the main use case of ANUP is where the AN and UPF are co-located even if they are separate functions. Therefore, the ANUP only absorbs the whatever functionalities that the separate UPF at the same site need to do anyway, with reduced signaling and data plane handling - the overall processing at the site actually decreases. While a particular ANUP now has more processing to do, it can offload some sessions to additional ANUPs that are now made possible because of removal of separate UPFs at the same site. This may also make it easier to allocate resources at the edge DC. Previously, an operator needs to consider how much resources to allocate for the separate UPFs and assign which sessions to which UPFs. Now it simply is to decide which sessions are assigned to which ANUP (just like to decide which sessions are assigned to which AN). In addition, there are some similar or even overlapping functionalities in the current UPF and AN in 5GS; in integrated ANUP these functions can be re-designed. For example for a rate control enforcement, UPF supports the enforcement of the aggregated MBR for the session (Session-AMBR) in UL/DL directions, while AN can enforce the aggregated MBR for the UE (UE-AMBR) in UL/DL directions. Both UPF and AN support the enforcement of the QoS Flow MBR (MFBR) and GBR (GFBR) in both UL/DL directions (for the GBR flows), while AN can in additon to ensure the UE-Slice-MBR is not exceeded in UL/DL directions. With ANUP, these previously separate functions may be optimized now that they are in the same entity.

Use of ULCL I-UPF for MEC Purpose Notice that the ANUP is to integrate AN and distributed UPF that are co-located in edge DCs, and one use case of distributed UPF (in those edge DCs) is MEC. UpLink CLassifier Intermediate UPF (ULCL I-UPF) is an existing way to achieve local breakout routing for MEC purpose, but it is not an optimized/elegant solution compared to ANUP. The ULCL I-UPF is placed between an AN and a central UPF as a filtering device. While called an UPF it is different from a typical UPF - It inspects all GTP-U UL traffic, and based on N4 signaling from SMF certain traffic is intercepted and forwarded to local DN. This places additional control plane burden on SMF in addition to the need of the special traffic-filtering UPF. For example, the SMF will need to know which traffic (to some particular destination address) is to be intercepted. For comparison, with ANUP there is no need for the additional special UPF and corresponding N4 signaling at all. Everything is standard routing/filtering w/o relying on SMF to determine which traffic is delivered locally:

For some PDU sessions, all traffic may be tunneled to a separate UPF.
For a particular PDU session, some traffic may be delivered locally while some other delivered to the central/remote DN all based on routing/filtering in the DN.

VPN PE Function in AN/ANUP As previously mentioned, the ANUP can optionally have the VPN PE function integrated, instead of being a standalone CE device for the VPN for the DN. While optional, it is a desired optimization. Moreover, even the separate AN itself can be considered as a spoke PE for a hub-and-spoke VPN for the DN. Consider a hub-and-spoke VPN outside the mobile network context:

A spoke PE only imports a default route from a hub PE and therefore sends all traffic from its CEs to the hub PE
A hub PE imports routes from all PEs and sends traffic to appropriate PEs or its CEs, whether the traffic is from a local CE or another PE

Additionally, consider that a spoke PE advertise different per-prefix (vs. per VRF) VPN labels. When it receives traffic with a per-prefix label, it can send traffic to a local CE purely based on the label without having to do a route lookup in the VRF. Now consider the AN and the central UPF in a mobile network. Effectively the AN is a spoke PE and the central UPF is a hub PE for the DN:

The GTP-U tunnel corresponds to the MPLS label stack.
For UL traffic, there is no need for route lookup on the AN because all is to be tunneled to the UPF. The UPF TEID is used by the UPF to determine which DN the traffic belongs to, just like how a VPN label is used to determine VPN the traffic belongs to.
For DL traffic, the UPF does a lookup based on the destination address (e.g., that of a UE) and a corresponding GTP-U tunnel is used to send traffic to an AN. When traffic arrives on the AN, the per-UE TEID allows traffic to be relayed to the UE without a route lookup.

In other words, the separate ANs and UPF form a hub-and-spoke VPN for the DN with per-prefix "labels", though no VRF is present on the ANs because there is no need for route lookup at all. For ANUP with VPN PE function integrated, the only difference is the addition of VRF in the AN. That's so that some sessions will be locally terminated and traffic is locally routed. For DL traffic, the ANUP can either advertise per-VRF label (or SID in case of SR) and do a lookup for DL traffic, or advertises per-prefix/UE label (or SID in case of SR) - just like per-UE TEID - so that it does not to do a lookup before sending traffic to a UE.

QoS Handling With separate AN and UPF, the QoS handling happens in the following segments:

Between UE and AN over the air interface
Between AN and UPF over the N3 tunnel, which can be:
- through a transport network, or
- through a local/internal link in co-location case

The QoS over the air interface is the same for both AN and ANUP cases. For the trivial QoS previously over N3 tunnel through a local/internal link in co-location case, it is now completely eliminated with ANUP. The QoS over N3 tunnel through a transport network is realized through QoS mechanisms in the transport network. With ANUP, it's likely that similar QoS is needed between the ANUP and a hub router in the DN, which is a VPN over the same transport network. Therefore, it is similar to the QoS over N3 tunnel - only that now it is QoS over VPN tunnel and realized through QoS mechanisms in the transport network. A central UPF may have rate limiting for N3 tunnels so that each PDU session's DL traffic is limited and the AN won't be overwhelmed by DL traffic. With distributed UPF (whether integrated into AN or not), the routes advertised to the hub DN router may carry QoS information like rate limiting parameters, so that the hub DN router can do rate limiting.

NAT Addresses assigned to UEs may be from a private address space and NAT is needed between the private space and public space. In case of central UPFs, the NAT can be done on a central UPF (though NAT is still a logically separate function) or by a separate NAT Gateway (GW) connected to the central UPF. With distributed UPFs (whether it is a separate UPF or an integrated ANUP), NAT can be done by a central NAT GW connected to the hub router, just like a NAT GW on or next to the previously central UPF. A large operator may have multiple central UPFs for different regions, and the regions may have overlapping private address spaces. Each UPF will have its own NAT GW, and UE to UE traffic across regions will go throw two NAT GWs. With distributed UPFs, each region will have its own hub router with its own NAT GW, and UE to UE traffic across regions will go through two NAT GWs and two hub routers.

Security Considerations To be provided.

IANA Considerations To be provided.

Acknowledgements The authors thank Arda Akman, Constantine Polychronopoulos, Sandeep Patel and Shraman Adhikary for their review, comments and suggestions to make this document and solution more complete.