The PCIe protocol (Peripheral Component Interconnect Express) had its first generation in 2003, being a huge breakthrough in the industry by allowing up to 2.5 GT/s per lane in a serial computer expansion bus. The protocol has since evolved many times, always doubling its transfer rate compared to the previous generation and bringing new features and optimizations whenever needed.
The latest release was announced in 2022, in which PCIe 6.0 was introduced with up to 64.0 GT/s speed per lane. As was announced at the PCI-SIG Developers Conference in San Jose, 2023, PCIe 6.0 not only again doubled the speed but also prepared the grounds for many generations to come. The changes were made considering many necessary optimizations to the existing rules, considering the industry usage and experience of 20 years. New concepts and technologies were introduced, such as 1b/1b encoding, PAM4 modulation, and Flit Mode operation.
For more information on Flit Mode, see Unraveling PCIe 6.0 Flit Mode Challenges.
Understanding PCIe 6.0 Optimizations
The changes in all features in PCIe 6.0 specification were done considering that they needed to be optimized to keep up with the higher throughput rate. Hence, the following guidelines were followed:
- Reduce loss: By avoiding unnecessary encoding;
- Make assumptions: Based on established patterns;
- Avoid transmitting unnecessary information: That could be inferred by the other side;
- Avoid reconfiguration: If it was already configured before.
1b/1b Encoding and Loss Reduction
The new encoding introduced in PCIe 6.0 is the biggest example of loss reduction by avoiding unnecessary encoding. Previously, instances used 128b/130b encoding when operating at 8.0 GT/s or higher speed. This means that every 128 bits of data required 2 extra bits in order to be correctly decoded by the other side. This caused an inefficiency in the serial link, in which 1.54% of the bandwidth was lost in the bit level simply due to encoding.
1b/1b addresses this problem by guaranteeing that every bit transmitted can be used as actual information by the other side. This is done by implementing internal counters in each side of what kind of data to expect. As long as the designs are correctly verified to respect those counters, it is guaranteed that they will be able to communicate without needing to send any extra unnecessary information in the link.
Flit Sequence Number and Optimizing Transmission of Information
The flit sequence number is a new concept introduced in PCIe 6.0, which was added together with the flit mode of operation. It replaces the old sequence number present in the Transaction Layer Packet (TLP), together with their acknowledgments or replay mechanisms.
Previously, sequence numbers were always attached to every TLP transmitted. Although it added robustness in the link, it turned out to be a waste of resources, considering that TLP had sequential sequence numbers. Therefore, knowing the sequence number of one TLP implied knowing the number of the TLP next to it, and so on.
The flit sequence number protocol optimized this by implementing the implicit sequence numbers, in which the sequence number is inferred by the other side. Not only that, but also the sequence number is in the flit level, which can accommodate many TLPs at once. Therefore, the space previously used for always transmitted sequence number information can be used to increase the bandwidth with useful information.
L0p and Optimizing Unnecessary Reconfiguration
Previously, the procedure to change the link width dynamically after link-up was costly for the devices since it required going through all the Configuration states of Link Training and Status State Machine (LTSSM). It meant reconfiguring all the details of the lanes, even though the only variable that required change was the link width being used.
This was enhanced in PCIe 6.0 with the introduction of L0 partial (L0p) feature, only present in Flit Mode. When the L0p sequence is performed, the link width can be changed during active data transfer, with no need to bring the link down. It means that performing power savings by changing the link width is much more effective and also, that devices can easily keep a smaller width in case they are having thermal throttling issues.
To know more about L0p, check Unraveling New Introduced PCIe 6.0 L0p.
In summary, PCIe 6.0 brought many changes, which were all optimizations to guarantee that all layers of PCIe protocol can keep up with the higher transfer rates. PCIe 7.0, which is currently in progress, continues the PCIe support and optimizations on top of all these changes. Therefore, it is very important to verify that devices follow the functional behavior of all those features to ensure they can take benefit from the advantages offered by the new protocol generation and also the versions that are yet to be announced.
More Information
- For more information on changes in PCIe 6.0, check What Disruptive Changes to Expect from PCI Express Gen 6.0
- For more information on verification of PCIe, check our VIP for PCIe and TripleCheck for PCIe
- For more information on PCIe in general, check PCI-SIG Website