CXL Overview hunglv0 18/03/24 Cache Protocol Channels • 3 channels in each direction: D2H vs H2D • D2H Requests from the device Cache Protocol Channels • H2D Requests are snoops from the host 2 Cache Protocol Channels • D2H Request: carries new requests from the Device to the Host • Each request will receive zero, one, or two responses and at most one 64-byte cacheline of data • D2H Response: carries all responses from the Device to the Host • Device responses to snoops indicate the state the line was left in the device caches, and may indicate that data is being returned to the Host • D2H Data: carries all data and byte enables from the Device to the Host. • A full 64-byte cacheline of data is always transferred • D2H Data may be temporarily blocked for link layer credits, but must not require any other D2H transaction 3 Cache Protocol Channels • H2D Request carries requests from the Host to the Device. • These are snoops to maintain coherency. Data may be returned for snoops. • The request carries the location of the data buffer to which any returned data should be written • H2D Response carries ordering messages and pulls for write data. • Each response carries the request identifier from the original device request to indicate where the response should be routed. • For write data pull responses, the message carries the location where the data should be written • H2D Data delivers the data for device read requests. • In all cases a full 64-byte cacheline of data is transferred 4 Device to Host Requests All device to Host CXL.cache transactions fall into one of these four semantics: • CXL.cache Read • CXL.cache Read0 • CXL.cache Read0/Write • CXL.cache Write 5 Device to Host Requests The MESI protocol is cache coherence protocol Modified (M) • The cache line is present only in the current cache, and is dirty - it has been modified (M state) from the value in main memory. The cache is required to write the data back to main memory at some time in the future, before permitting any other read of the (no longer valid) main memory state. Exclusive (E) • The cache line is present only in the current cache, but is clean - it matches main memory. Shared (S) • Indicates that this cache line may be stored in other caches of the machine and is clean - it matches the main memory. The line may be discarded (changed to the Invalid state) at any time. Invalid (I) • Indicates that this cache line is invalid (unused). 6 Device to Host Requests D2H Read Request Message 7 Device to Host Requests D2H Read0 Request Message 8 Device to Host Requests D2H Write Request Message 9 Host to Device Requests The host will send the following 3 types of snoop requests: • SnpData • SnpInv • SnpCur 10 Host to Device Requests H2D Snoop Request Message 11 Memory Protocol CXL.mem is a transactional interface between the CPU and Memory The CXL.mem provides 3 basic coherence models for CXL.mem Host-managed Device Memory (HDM) address regions exposed by the CXL.mem protocol: • HDM-H (Host-only Coherent): Used only for Type 3 Devices • HDM-D (Device Coherent): Used only for legacy Type 2 Devices that rely on CXL.cache to manage coherence with the Host • HDM-DB (Device Coherent using Back-Invalidate): Can be used by Type 2 Devices or Type 3 Devices 12 Memory Channel • The CPU coherency engine is regarded as the CXL.mem Master. • The Mem device is regarded as the CXL.mem Subordinate. 13 Memory Channel M2S Request: contains reads, invalidates, and signals going from the Master to the Subordinate • • • • • • • • • • • • MemInv MemRd MemRdData MemRdFwd MemRdFwd MemRdTEE MemRdDataTEE MemSpecRd MemInvNT MemClnEvct MemSpecRdTEE TEUpdate 14 Memory Channel The Request with Data (RwD): contains writes from the Master to the Subordinate. • • • • • • • MemWr MemWrPtl BIConflict MemRdFill MemWrTEE MemWrPtlTEE MemRdFillTEE 15 Memory Channel The Back-Invalidate Snoop (BISnp): contains Snoop messages from the Subordinate to the Master. This message class is not supported in 68B Flit mode. • • • • • • BISnpCur BISnpData BISnpInv BISnpCurBlk BISnpDataBlk BISnpInvBlk The Back-Invalidate Response (BIRs: contains response messages from the Master to the Subordinate as a result of Back-Invalidate Snoops. This message class is not supported in 68B Flit mode. • • • • • • BIRspI BIRspS BIRspE BIRspIBlk BIRspSBlk BIRspEBlk 16 Memory Channel The No-Data Response (NDR) message class contains completions and indications from the Subordinate to the Master. • • • • • • Cmp Cmp-S Cmp-E Cmp-M BI-ConflictAck CmpTEE 17 Memory Channel The Data Respons (DRS): contains memory read data from the Subordinate to the Master. • MemData • MemData-NXM • MemDataTEE 18 Thanks for your attention Dolphin Technology Vietnam Center 2nd Floor, Lilama Building, 124 Minh Khai, Hai Ba Trung Dist., Hanoi +84 243 624 9784 https://www.dolphin-vc.com jobs@dolphin-vc.com
