Flit Mode - Complete Guide

PCIe 6.0/7.0's Fundamental Architecture Change

64 GT/s & 128 GT/s Operation
Home → Flit Mode Complete Guide

Why Flit Mode?

PCIe 6.0 introduced Flit Mode as a fundamental architectural change to address the challenges of operating at 64 GT/s and beyond with PAM4 signaling.

The PAM4 Challenge

PAM4 signaling has inherently higher bit error rates than NRZ due to smaller voltage margins (3 eyes vs 1). The legacy per-TLP CRC approach would result in unacceptable packet loss rates at 64+ GT/s.

Solution: Fixed-size Flits with integrated FEC (Forward Error Correction) to correct errors before they cause packet loss.

Non-Flit Mode vs Flit Mode

AspectNon-Flit Mode (Legacy)Flit Mode (6.0+)
Packet SizeVariable (up to 4KB payload)Fixed 256-byte Flits
Error ProtectionLCRC per TLPFEC + CRC per Flit
Error HandlingDetect & retransmitCorrect in-line, retransmit if needed
Encoding8b/10b or 128b/130b1b/1b PAM4
Max Tags10-bit (1024)14-bit (16384)
DLLPsSeparate packetsEmbedded in Flit DLP

Flit Structure

A Flit (Flow Control Unit) is exactly 256 bytes, providing a fixed container for TLPs and control information.

256-Byte Flit Structure Flit Hdr 2 bytes TLP Payload Area Up to 236 bytes (multiple TLPs possible) DLP 6-8 bytes FEC + CRC 8 bytes 0-1 2-237 238-245 246-255 Total: 256 bytes exactly
// Flit Header (2 bytes) Byte 0: [7:6] Flit Type - 00=Payload, 01=NOP, 10=Ack, 11=Nak [5:0] TLP Count - Number of TLPs starting in this Flit (0-6) Byte 1: [7:0] Flit Seq Num - 8-bit sequence number (0-255, wraps) // DLP (Data Link Payload) - embedded flow control & ACK info Contains: - Credit updates (replaces UpdateFC DLLPs) - Implicit ACK information - Sequence acknowledgment // FEC + CRC (8 bytes) - 6 bytes FEC parity (can correct burst errors up to ~48 bits) - 2 bytes CRC for error detection after FEC

Flit Types

Payload Flit (Type 00)

Contains TLP data. Multiple small TLPs can be packed into one Flit, or large TLPs span multiple Flits.

NOP Flit (Type 01)

Idle Flit sent when no TLP data available. Still carries DLP for flow control. Subtypes: NOP.Empty, NOP.Debug, NOP.Vendor.

ACK Flit (Type 10)

Explicit acknowledgment of received Flits. Contains sequence number being acknowledged.

NAK Flit (Type 11)

Request retransmission. Contains sequence number of missing/errored Flit.

TLP Packing in Flits

Multiple TLPs can be packed into a single Flit, and large TLPs span across Flit boundaries.

// TLP Packing Rules 1. Up to 6 TLPs can start in a single Flit 2. TLP can span Flit boundaries (continued in next Flit) 3. Small TLPs (headers only) efficiently packed 4. Padding bytes fill unused space // Example: Multiple small TLPs in one Flit Flit: [Hdr][TLP1-CfgRd(12B)][TLP2-Cpl(12B)][TLP3-MRd(16B)][Padding][DLP][FEC] // Example: Large TLP spanning Flits Flit N: [Hdr][TLP Start - Header + 220B data][DLP][FEC] Flit N+1: [Hdr][TLP Continue - remaining data][Next TLP...][DLP][FEC]

Forward Error Correction (FEC)

FEC is mandatory in Flit Mode and enables correction of bit errors without retransmission.

FEC Capabilities

  • Error Correction: Corrects burst errors up to ~48 bits per Flit
  • Latency Impact: Adds ~2-3 ns encoding/decoding latency
  • Algorithm: Based on Reed-Solomon codes
  • CRC Verification: After FEC decode, CRC verifies correction was successful

If FEC cannot correct errors (too many), the Flit fails CRC check and triggers NAK/retry.

Flit Error Rate ≈ (Raw BER)^(FEC_correction_capability+1)
Example: 10^-6 BER with 48-bit correction → effectively ~10^-12 Flit error rate

Flow Control Changes

Flit Mode changes flow control granularity from 16-byte FC units to 256-byte Flits.

Credit Changes

  • Credit Unit: 1 credit = 1 Flit (256 bytes) instead of 4 DW (16 bytes)
  • UpdateFC: Embedded in Flit DLP, not separate DLLPs
  • Simpler Accounting: Fixed Flit size simplifies calculations

TLP Header Format Changes

Flit Mode optimizes TLP headers for efficiency.

8-Bit Type Field

Combined Fmt+Type into single 8-bit field. More encodings available for future expansion.

14-Bit Tags

Contiguous 14-bit Tag field supports 16,384 outstanding transactions.

OHC Field

5-bit Optimized Header Compression field indicates header compression mode.

No TH Bit

TPH (TLP Processing Hints) handled differently. TH bit removed from header.

Optimized Header Compression (OHC)

OHC reduces header overhead by allowing abbreviated headers when fields can be inferred.

// OHC Types (5-bit field) OHC-A: Requester ID compression - OHC-A1: Implicit Requester ID (same as previous TLP) - OHC-A2: Partial Requester ID - OHC-A3: Full Requester ID + Segment (for Config requests) OHC-B: Completion Header compression - Byte Count and Lower Address can be implicit OHC-C: IDE (encryption) related fields OHC-D: Tag compression (for sequences) OHC-E: Extended compression modes

14-Bit Tags

Flit Mode enables 14-bit Tags, dramatically increasing the number of outstanding transactions.

Tag SizeMax OutstandingModeEnable Bit
5-bit32BothDefault
8-bit256BothExtended Tag Enable
10-bit1,024Both10-Bit Tag Requester Enable
14-bit16,384Flit Only14-Bit Tag Requester Enable

ACK/NAK in Flit Mode

The ACK/NAK mechanism is adapted for Flit-level operation.

// Flit Mode ACK/NAK Implicit ACK: - Every received Flit implicitly acknowledges previous Flits - DLP contains latest acknowledged sequence number Explicit ACK Flit: - Flit Type = 10 - Contains sequence number being acknowledged - Used when no Payload Flits to send NAK Flit: - Flit Type = 11 - Triggers retransmission from specified sequence - Sent when FEC fails to correct errors 8-bit Sequence Numbers: - Range 0-255 (vs 12-bit in Non-Flit Mode) - Simpler tracking with fixed Flit sizes

L0p Power State

Flit Mode introduces L0p, a new power state for partial bandwidth operation.

L0p (L0 Partial)

Allows reducing active lanes while maintaining L0 operation. Unlike L0s/L1, traffic continues on remaining lanes.

  • Lane Reduction: x16 can reduce to x8, x4, x2, or x1
  • Fast Transition: ~100ns to change width
  • Continuous Operation: No entry/exit latency like L0s
  • Power Savings: Linear with reduced lanes

Flit Mode Negotiation

Flit Mode is negotiated during link training.

// Flit Mode Negotiation (in Training Sequences) TS1/TS2 Fields: - Flit Mode Supported bit - Flit Mode Requested bit (64 GT/s requires Flit Mode) Rules: - Both ends must support Flit Mode for it to be enabled - 64 GT/s and above REQUIRE Flit Mode (cannot use Non-Flit) - 32 GT/s and below can use either mode - Mode determined during Configuration state Data Stream Selection: - Non-Flit: SDS (Start of Data Stream) followed by DLLPs/TLPs - Flit: SDF (Start of Data Flit) followed by Flits
Back to Index →