| Internet-Draft | Cycle Mapping Learning Method for Scalin | May 2023 |
| Zhu, et al. | Expires 26 November 2023 | [Page] |
The scaling DetNet (Deterministic Networking) technology based on cyclic queuing and scheduling is expected to solve the scalability problem of DetNet, and is hoped to extend the adaptive domain of DetNet to wide area network or even backbone network. One of the keys of this technology is to accurately obtain the cyclic mapping relationship between adjacent nodes, based on which DetNet packets can be end-to-end deterministically forwarded . This draft proposes a method for nodes to learn the cycle mapping relationship through sending learning messages.¶
This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶
Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶
Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶
This Internet-Draft will expire on 26 November 2023.¶
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.¶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶
[I-D.ietf-detnet-scaling-requirements] proposes the requirements for deterministic forwarding of services in scaling DetNet(Deterministic Networking) networks. Typical characteristics of large-scale networks include long single-hop delay, no time sync, and large network dimensions (typically, greater than 16 hops), etc., these requirements cannot be met by the DetNet architecture and technology proposed by [RFC8655], because the architecture andtechnology proposed by RFC8655 are mainly intend for limited-scale networks. In order to provide deterministic services in large-scale networks, the industry has proposed a variety of improving ideas, such as [SR-TSN], [Deadline] and [TCQF], and one of the promising and first-deployed solutions is based on the idea of cyclic scheduling and forwarding from [IEEE802.1Qch].¶
In this draft, for two adjacent nodes, one is called upstream node (UN) and the other downstream node (DN). The cycle-mapping relationship refers to the correspondence between the cycle label carried by the upstream node`s egress packet and the cycle label encapsulated by the downstream node`s egress port. The packet is forwarded under the guidance of the cycle mapping relationship. For example, as shown in Figure 1, if there is a cycle mapping relationship x->z between UN and DN, when the packet with the cycle label x arrives at the downstream node , the downstream node will obtain the downstream egress cycle label according to the mapping relationship x->z, then z will be encapsulated into and the packet will be scheduled and forwarded according to the cycle slot z.¶
| cycle x | |
UN +-----------+-----------+
\
\ Packet
\ received with x
| V | | cycle z |
DN +-----------+ ... ... +-----------+
\
\ Packet
\ sent with z
V
To obtain the cycle mapping relationship, we have many optional methods, for example, one can use control plane for calculation and configure the result to forwarding node, and one can also leave the task to forwarding plane self-learning. In the forwarding plane learning mode, cycle mapping learning messages are exchanged between upstream node and downstream node, and the downstream node have to calculate and store the cycle mapping relationship according to the learning messages.¶
This draft gives the general idea, problems and corresponding solutions for cycle mapping self-learning. This draft only considers the general learning process of the forwarding plane , and does not consider the underlying protocol (e.g., OAM) and protocol extension format of the learning message, nor does it consider the interaction protocol extension between the control plane and the forwarding plane. The consideration of related extensions will be described in detail in future drafts.¶
UN: Upstream Node¶
DN: Downstream Node¶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶
The cycle mapping learning process requires UN to send learning messages to DN. No restriction is put on the detailed encapsulation format, the sending frequency, sending times and how the downstream node can identify the message, for example, it can be configured by the control plane.¶
At least three optional cycle mapping learning message types can be used.¶
In order to clearly mark the egress cycle slot boundary, the UN can constructs and sends special protocol messages (such as PAUSE, TWAMP) as learning messages at the beginning and the ending of the cycle. In order to avoid occupying too much bandwidth, special messages are sent at a reasonable frequency, for example, sent once a physical cycle. The start time of cycle learning may be the time of system startup, controller configuration time, user triggering and so on. When a learning message arrives at a DN, it can be used to clearly mark the boundary of the UN egress cycle at local, which is called a UN virtual cycle.¶
Advantages: easy to mark cycle boundaries, out-of-band, flexible and controllable;¶
Disadvantages: Bandwidth consumption is required, and special message construction brings hardware overhead;¶
Before DetNet flow enters the network, the cycle mapping relationship does not work and is meaningless. At the same time, in order to avoid bandwidth consumption in the active mode, learning can be based on actual DetNet flow packets, because when deterministic packets are sent from the source node, they will inevitably carry the egress cycle label of the UN egress. There is no restriction on the packet type, for example, it may be an SRv6 extended packet or an SR-MPLS extended packet.¶
Advantages: In-band mode can realize packets multiplexing;¶
Disadvantages: The number and time of sending actual flow packets are uncertain, resulting in uncertain positions of flow packets within the cycle, and cannot be directly used to calibrate the cycle boundary;¶
In this manner, the field of the flow packet is extended, and the value of "deviation from the beginning of the cycle slot" is additionally encapsulated, which can be used to clearly identify the position of the flow packet in the UN egress cycle, thereby assisting the DN to determine the virtual cycle boundary where the flow packet is located.¶
Advantages: In-band mode, multiplexing can be realized, and downstream nodes are easy to learn;¶
Disadvantage: The purpose of the extra encapsulation field is only for periodic learning, occupying the effective field of the business message. Both encapsulation and decapsulation logic need to be modified, increasing hardware resource consumption.¶
According to specific scenarios, one of the above methods can be selected for cycle mapping learning messages.¶
The cycle mapping self-learning process is carried out by the DN, relying on the cycle mapping learning message sent by the UN. It is assumed that the number of queues and cache capacity configured by the UN are sufficient. The functions of packet routing and forwarding are independent from learning process described here and will not be described.¶
According to the latency reference model proposed in the [RFC9320], the node regulation subsystem must be able to absorb the maximum node processing delay. Typically, the processing delay variation scope only depends on the hardware implementation of a specific device, including shaping, table lookup algorithm, the packet counter resources and etc. For a specific network device, the maximum value of processing delay is determined at the product, and usually includes two components, namely the packet length related part and packet length independent part. For instance, packet-length-independent part include forwarding table lookup, CRC check, etc., and the packet length-related part includes packet cache and scheduling and so on. If the processing delay is known and can be queried by the self-learning module, no active OAM probing is required.¶
Though processing delay may depend on enabled functions (such as shaping, packet slicing) and packet length, the processing delay range can also be obtained through some active detection tools. Specifically, for a given device with enabled function sets, one can randomly send packets that meet the packet length limit and up to wire-speed, and then it is easy to measure the maximum processing delay. Further, the delay value can be configured in a device-specific field , and then can be used by the cycle mapping self-learning module.¶
The processing delay can also be configured by the control plane as needed. For example, when the control plane needs to control the end-to-end delay of the DetNet domain, it may actively increase the single-hop process delay of the device.¶
Suppose that the special purpose cycle learning message described in Section 2.1 is sent by UN, it is easy for DN to correctly identify the boundary of UN egress virtual cycle slot. Different from the IEEE802.1Qch mechanism, since the upstream and downstream nodes are not time synchronized and there are long links, the packets carrying the same UN egress cycle label x, may not necessarily fall into the same DN cycle y-1, in fact some packets may also fall into cycle y, as shown in Figure 2.¶
| cycle x | |
UN +-----------------+-----------------+
\ \
\ \
\ learning msg \learning msg
\ P1 received \P2 received
\ in cycle y-1 \in cycle y
| V | V | |
DN +-----------------+-----------------+-----------------+
y-1 y y+1
In order to correctly obtain the egress cycle number of the downstream node, it is necessary to take the arrival time of the latest packet belonging to UN egress cycle x as the baseline, that is, y as the final receiving cycle. In the implementation, it is necessary to have a table storing the cycle mapping relationship between the UN egress and the DN ingress cycle label. When the first learning message P1 carrying the cycle label x is received, the mapping relationship x->y-1 is obtained and stored. When the second learning message carrying cycle label x is received , UN will obtain x->y relationship and update the table.¶
The DN calculates the local ingress->egress port cycle mapping relationship according to the maximum processing delay obtained in Section 4.1, as shown in Figure 3.¶
UN | cycle x | |
egress +-----------------+-----------------+
\ \
\P1(x) \P2(x)
\ \
V V
<--virtual cycle-->
DN ingress+-----------------+--*--------------+ ....+-----------------+
| | * | | |
| | <-max process delay->| |
| cycle y-1 | cycle y | | * cycle z |
DN egress +-----------------+-----------------+ ....+-*---------------+
\ \
\P1(z) \P2(z)
\ \
V V
The calculation method of the egress cycle label of DN is no very simple and intuitive, it needs to consider the cycle slot size, the slot offset of the P2 message falling in the cycle and the maximum processing delay of UN at the same time. Assuming that the cycle slot size is T and the receiving cycle is y, the formula is as follows:¶
UN egress cycle z=y+ ceil((processing delay-T+offset)/T) + 1¶
In formula above, ceil means to round-up the division result.¶
In particular, if (processing delay-T+offset) happens to be an integer multiple of T, in order to avoid the last flow packet of cycle x sent to thez queue cannot be forwarded in time and cause an error, because, for example, hardware implementationprecision, z can be proactively increased by 1 for redundant configuration .¶
Then can get the DN ingress and egress cycle mapping relationship y->z.¶
After obtaining the mapping relationship x->y from the upstream egress to the downstream ingress and the mapping relationship y->z from the downstream ingress to the egress, the mapping relationship x->z can be easily obtained. Since there are multiple circular queues, assuming that the number of circular queues of template T is N, the final cycle mapping relationship can be further obtained:¶
(x+i) MOD N -> (z+i) MOD N¶
For example, the UN and DN concurrently maintain a eight-cycle-queue template. Through learning, the cycle mapping relationship between the upstream egress and the downstream egress port is 4->6, then the mapping relationship maintained at the DN is:¶
4->6, 5->7, 6->0, 7->1,0->2,1->3,2->4,3->5.¶
Among the three cycle self-learning messages mentioned in Section 2, both 3.1 and 3.3 can accurately identify the virtual cycle boundary where the message is located at the downstream node. When learning based on flow packets proposed in 3.2, since the arrival frequency and time of flow packets are random in cycle slot, flow packets cannot be directly used to accurately identify cycle boundaries.¶
Considering the scenario shown in the figure 4 upper half, since all packets of cycle 1 arrive at the downstream node, they all fall in the cycle 3, so the mapping relationship of UN egress and DN ingress, 1->3 is formed. In Figure 4 lower half, when the number of service packets increases, some packets fall into cycle 4. According to Section 4.2, it must be considered that the packet receiving cycle is 4, and the mapping relationship is adjusted to 1->4.¶
| cycle 1 | |
UN +-----------------+-----------------+
\ \ \ \
\ \ \ \
\ \ \ \
\ \ \ \
\ \ \ \
| V V V V | | |
DN +-----------------+-----------------+-----------------+
3 4 5
| cycle x | |
UN +-----------------+-----------------+
\ \ \ \\ \ \ \
\ \ \ \\ \ \ \
\ \ \ \\ \ \ \
\ \ \ \\ \ \ \
\ \ \ \\ \ \ \
| V V V VV| V V V | |
DN +-----------------+-----------------+-----------------+
3 4 5
Which add a T to the single-hop jitter after adjustment in figure 4 . From the perspective of the entire service flow, the end-to-end jitter requirement of 2T cannot be met.¶
Idea 1 . When learning the mapping relationship for the first time, if all the UN egress cycle packets fall into the same DN ingress cycle, for example, fall into cycle y, then take the automatic adjustment and supposed it to fall into cycle y+1. At this time, by introducing a redundancy delay of T, the uncertainty introduced by flow packets can be avoided . The end-to-end delay increases by N*T, where N is the number of hops, and the max end-to-end jitter is still 2T.¶
Idea 2. Reshape the business message, and if there is more than flow packets in the cycle, since the queue is a FIFO, there will always be a flow packets to be sent at the beginning of the cycle, so only one message needs to be reserved and be used to mark the end of the cycle There is always a message at every moment. This method does not affect packet forwarding characteristics and bandwidth occupation.¶
Idea 3. The network construction is divided into two phases: testing and operation. When the flow packets requests access, it is required to send test flow packets at random or full flow for a predefined period. After the period, actual DetNet flow packets are sent according to the requirements of the traffic characteristics.¶
After the cycle mapping relationship is established, flow packets can directly search the mapping table to obtain the egress cycle label, and then deterministic forwarding can be obtained. However, when some problems occur, for example, the fiber switching occurs , forwarding node restarts, link attenuation occurs, or the learning process is re-triggered by the control plane or user, the mapping relationship of the upstream and downstream nodes may change. If such anomalies are not detected in time, errors will occur in the cycle mapping process, and DetNet flow determinism cannot be guaranteed.¶
In order to detect changes in time and update the mapping relationship,for each receiving flow packets, the DN still needs to detect ingress cycle, and compare it with the cycle mapping relationship maintained in Section 4.4. If it is inconsistent, it indicates that cycle mapping relationship has changed. For example, if the mapping relationship 2->5 is maintained in 4.4, when a UN flow packet is received and the ingress cycle of the DN is 4 or 5, it is considered correct, and in other cases it is considered problems has take place, it is necessary to re-execute the cycle mapping relationship calculation process described in Sections 4.¶
Consider the situation where multiple upstream nodes establish a cycle mapping to a single downstream node . When the downstream node maintains the mapping relationship, it needs to ensure the uniqueness of the table key value.¶
i.For point-to-point links, you can simply use {DN ingress port, DN egress port} as the key value;¶
ii.In the case of LAN (many-to-many or many-to-one) , multiple UN may reach Dn through the same ingress port, as shown in the figure 5, at this time, the downstream node entry is the same for multiple upstream nodes and cannot be used as a key value. Considering that in routing and forwarding, the MAC address of the upstream node will be carried in the message, so the MAC address can uniquely identify the upstream node. At this time, the UN egress source mac can be added to the key value:{mac, DN ingress port , DN egress port}.¶
/-----\
| UN1 | ________
+--+--+ _____ / \
| / \__/ \
| / \_____
| / \_____
+-->| \
| |
+-->| LAN |
| \ | /-----\
| \ |------>| DN1 |
| \ | +--+--+
/-----\ \ /
| UN2 | \___ /
+--+--+ \ /
\_____/ \___________/
If the upstream and downstream nodes support multiple cycle templates at the same time, for each cycle template, a mapping relationship can be formed through cycle self-learning process described in this draft, and the process of cycle mapping self -learning is exactly the same as that of a single cycle template.¶
In order to identify the periodic template at the downstream node, in addition to the time slot number, the periodic template T needs to be encapsulated in the learning message . The downstream node first extracts T and then performs learning based on the corresponding template. In addition, after the periodic self-learning calculation is completed, the entry key needs to be added with a template when storing, that is, {T, mac, DN ingress port , DN egress port} .¶
TBA¶
TBA¶