Interrupt Mechanisms
PCIe supports multiple interrupt mechanisms for backward compatibility and modern efficiency:
| Mechanism | Vectors | Routing | Recommendation |
|---|---|---|---|
| INTx (Legacy) | 4 (shared) | In-band messages | Avoid - legacy only |
| MSI | 1-32 | Memory writes | Preferred for simple devices |
| MSI-X | Up to 2048 | Memory writes | Preferred for complex devices |
MSI and MSI-X
Why MSI/MSI-X is Better
- No shared interrupt lines
- Multiple vectors per device
- Lower latency - memory write vs. sideband signal
- Per-CPU targeting possible
- No IRQ routing complexity
MSI-X Table Structure
MSI-X uses a table in device memory (BAR space) with entries containing:
- Message Address (64-bit)
- Message Data (32-bit)
- Vector Control (mask bit)
Error Handling
Error Classification
- Correctable: Hardware can recover, no data loss
- Non-Fatal Uncorrectable: Transaction failed, but link OK
- Fatal Uncorrectable: Link compromised, reset needed
| Error Type | Examples | Reporting |
|---|---|---|
| Correctable | Receiver Error, Bad TLP, Bad DLLP, Replay Timeout | ERR_COR message |
| Non-Fatal | Completion Timeout, Unsupported Request | ERR_NONFATAL message |
| Fatal | Data Link Protocol Error, Malformed TLP, ECRC Error | ERR_FATAL message |
Advanced Error Reporting (AER)
AER provides detailed error logging and control through extended capability registers:
- Individual error masking
- Error severity programming
- Header logging for failed TLPs
- First error pointer
- Root Complex error aggregation
Hot-Plug Support
PCIe supports native hot-plug for adding/removing devices without system shutdown:
Attention Indicator
LED showing slot status (on, off, blinking)
Power Indicator
LED showing power status to slot
Attention Button
User request for hot-plug operation
Power Controller
Hardware to control slot power
Presence Detect
Detects card insertion/removal
MRL Sensor
Manually-operated Retention Latch sensor
Access Control Services (ACS)
ACS provides isolation controls for peer-to-peer transactions, critical for virtualization:
- ACS Source Validation: Validates Requester ID
- ACS Translation Blocking: Blocks translated requests
- ACS P2P Request Redirect: Redirects P2P to Root Complex
- ACS Upstream Forwarding: Controls upstream forwarding
- ACS Direct Translated P2P: Controls direct P2P for translated
AtomicOps
AtomicOps provide atomic read-modify-write operations across PCIe:
- FetchAdd: Fetch and Add
- Swap: Unconditional Swap
- CAS: Compare and Swap
Useful for synchronization in multi-device systems (GPUs, accelerators).
Downstream Port Containment (DPC)
DPC automatically contains errors at a downstream port to prevent propagation:
- Disables the downstream port on fatal error
- Prevents error storm from affecting system
- Allows software recovery