In continuation of our series on IDE blogs, Why IDE Security Technology for PCIe and CXL? and Verification of Integrity and Data Encryption(IDE) for PCIe Devices, this blog focuses on IDE verification considerations for CXL devices.
With increasing trends of AI, ML, and deep learning in the computing space, there is a focus on security features for SoCs catering to high performance computing (HPC), data analytics, automotive, etc. CXL is rapidly growing as the interconnect of choice for these applications, which does pose security concerns while transmitting mission-critical workloads. Therefore, CXL specification decides to incorporate security IDE features with a similar flow as PCIe IDE. In fact, the operational modus for FLITs transmitted on CXL.io semantics is the same as PCIe.
Coming to CXL.cachemem, AES-GCM is used with 256 key sizes for data encryption and integrity and 96-bit MAC for data protection. However, CXL.cachemem supports only Link IDE. Let’s take a closer look at key verification aspects for CXL.cachemem IDE.
Functional Considerations
MAC Generation and Handling
Unlike CXL.io/PCIe IDE, MAC generation for CXL.cachemem IDE is done over an aggregated number of multiple FLITs known as MAC Epoch window. The numbers of FLITs aggregated together depend on mode of operation, i.e, Containment mode (5) and Skid mode (128). Ensuring the correctness of MAC transmitted and received includes making sure keys are corrected and communicated between two components, invocation vector is incremented appropriately at Epoch windows, the accuracy of AES-GSM algorithm, the IDE configuration settings are configured, etc. So, having the first testcase up with Memory Read/Write transaction with IDE enabled itself is a major milestone from a verification standpoint. Notably, specification does allow TX and RX paths to operate in different IDE modes. However, this must be enabled only if the application demands so.
Key Programming and Exchange
Specification mentions key programming, and exchange is implementation specific. Designs might adopt application-specific registers for key programming and use sideband signaling, SPDM (Security Protocol and Data Model), or PCIe vendor-defined messages to exchange keys over the link. Sanity tests around how keys are exchanged must be exercised. Also, consider scenarios to emulate behavior for security threats on key exchange mechanism resulting in keys being leaked. Specification also defines the usage of alternative keys. As long as it doesn’t change in the middle of a MAC Epoch window the designs are open to implement key toggles based on time, interrupt, or more sophisticated algorithms. Testcases around key toggles are a must to ensure encrypted link operation post key toggles as well.
Early MAC Termination
An interesting verification aspect that is peculiar to CXL.cachemem is possible early termination of MAC. Since MAC is transmitted for ‘N’ number of flits depending on the IDE mode of operation, there could be scenario where there is no continuous stream of FLITs from upper layers. In such cases, the specification provides a provision to terminate MAC_Epoch window earlier and send idle frames. Testcases having large numbers of packets exercising various delays, surpassing numerous Epoch windows with sandwiched control FLITs are bound to expose corner-case bugs. The below snapshot from CXL specification illustrates the scenario for early MAC termination.
Plaintext CRC Testing
Plaintext CRC is an optional feature, which is another test vector to be considered as a random setting. Unlike CXL.io/PCIe IDE, PCRC is not transmitted over link for CXL.cachemem semantics. Below are some snapshots from CXL specification for PCRC inclusion during encryption and decryption.
Performance Considerations
Post verifying the functional correctness of IDE encrypted link, it is a must for verification engineers to take a closer look at performance/latency numbers. Skid mode, by design, should have a lower penalty on latency and bandwidth numbers than containment mode. Skid mode does allow near-zero latency overhead. Containment mode latency is added due to the requirement of ensuring MAC integrity before releasing the FLITs for further processing. Skid mode guarantees better bandwidth utilization as MAC is transmitted after 128 FLITs, while for containment mode, MAC is transmitted for every 5 FLITs.
As established by now, verifying CXL IDE requires a robust and exhaustive approach. Cadence PCIe Verification IP and TripleCheck solution can help significantly reduce verification project cycles with comprehensive APIs, checkers for protocol violation and test suite.
More Information
- For more info on how Cadence PCIe Verification IP and TripleCheck VIP enables users to confidently verify IDE , see our VIP for PCI Express, VIP for Compute Express Link and TripleCheck
- For more information on PCIe in general, and on the various PCI standards, see the PCI-SIG website
- For more information on CXL in general, see CXL consortium website