Introduction
Unordered IO (UIO) ECN is included in the PCIe 6.1 specification and defines a new wire semantic and related capabilities for addressing the limitations of the PCI/PCIe fabric-enforced ordering rules. The key motivations for UIO ECN are to enable multiple-path fabric support and to have performance improvements.
Previous Ordering Methods vs UIO
In the PCIe protocol, there are some transaction ordering rules to avoid deadlocks and producer-consumer problems. For example,
- A posted request must not pass another posted request
- A posted request must be able to pass non-posted requests to avoid deadlocks
- Non-posted requests with data must not pass a posted request
These rules are defined for all types of transactions on PCI Express, including memory, I/O, configuration, and messages. The ordering rules are applicable for transactions with the same Traffic Class (TC).
The PCIe specification provided a way to change the ordering rules by using Relaxed Ordering (RO) and ID-Based Ordering (IDO) attributes present in the TLP. Header attributes bits [2:1].
Table 1: Ordering attributes
Nowadays, however, many new architectures have introduced fabric with multiple paths available between source and destination, and hence, earlier efforts to relax the PCIe ordering rules, such as RO and IDO, do not fully address the challenges.
Unlike producer-consumer ordering, in which semantics are enforced at every entity, and all enforce the same ordering semantics, UIO moves the producer-consumer enforcement to the source. This helps to avoid unnecessary traffic and enables parallel paths to deliver better bandwidth and latency.
What Is New for UIO?
First, several new TLP types are defined for UIO transactions, which includes UIO Mem Read (UIOMRd), UIO Mem Write (UIOMWr), UIO Read Completion (UIORdCpl and UIORdCplD), and UIO Write completion (UIOWrCol). Previously, Posted packets like MemWr didn’t need completion, but all UIO requests require completions. Hence, UIO write requires completion to complete the packet cycle.
UIO is only defined for Flit Mode, and no translation of UIO TLPs to Non-Flit Mode (NFM) is permitted. The below table shows newly defined TLP types for UIO requests and completions.
Table 2: TLP types for UIO requests and completions
Second, UIO TLPs must only be transmitted on Virtual Channels (VCs) that are configured for UIO. Hence, at least two VCs are required to support UIO because VC0 is always reserved for non-UIO traffic, and VC3 is required to use UIO. Ports supporting UIO must implement a new extended capability, the Streamlined Virtual Channel (SVC). SVC provides explicit support for architecturally defined TC/VC applications via a combination of hardware requirements and software guidance. The TC/VC default HW assignments in the table below are mandatory for SVC.
Table 3: TC/VC default HW assignments
The SVC capability is optional for devices supporting PCIe 6.1 but is mandatory if supporting capabilities associated with this structure, including UIO. It can be supported together with VC or MFVC capabilities, but only one can be enabled at a time.
Third, the transaction ID filled in the TLP will now carry the TC information for UIO requests. The requirement is to have the transaction ID consisting of the TC[2:0], Requester ID[15:0] and Tag[13:0], and 14-bit tags are mandatory to be supported for UIO completers.
Fourth, to indicate support for UIO, two new bits are defined in the device capabilities 3 register: [10] UIO Mem RdWr completer supported and [11] UIO Mem RdWr requester supported. Besides those, bit [7] UIO Mem RdWr requester enable is used to enable the functionality.
Lastly, and most importantly, ordering rules changes for UIO are also well explained in 6.1 Specification, including that UIO Completions must be allowed to pass UIO Requests and that UIO allows arbitrary reordering for all other cases. Spec also defines that UIO and non-UIO TLPs are never mixed within a TC/VC, and ordering dependencies between UIO and non-UIO TLPs is not permitted.
Design Consideration and Verification Challenges
With all the new features described above, implementers need to fully consider all new UIO TLP types, especially completions for the UIO write scenario. Also, new SVC capability structures are mandatory, and new VC usage should be well implemented. The new ordering rules will be the most important functionality and should be tested on all possible scenarios and combinations to make sure there is no deadlock for UIO requests.
More Information
- For more info on how Cadence PCIe Verification IP and TripleCheck VIP enable users to confidently verify PCIe 6.1, see VIP for PCI Express, VIP for Compute Express Link, and TripleCheck for PCI Express
- See the PCI-SIG website for more details on PCIe in general and the different PCI standards.