CTI_010

The time CSR MUST increment at a constant frequency and the count MUST be in units of 1 ns. The frequency at which the CSR provides an updated time value MUST be at least 100 MHz.

CTI_020

The time counter MUST appear to be always on and MUST appear to not lose its count across hart low power idle states, including when the hart is powered off.

IIC_010

The RISC-V Advanced Interrupt Architecture [8] MUST be supported.

IIC_020

External interrupts MUST be signaled to a hart as message-signaled interrupts (MSI).

Since Message Signaled Interrupts (MSI) are implemented as memory writes, they facilitate a simplified enforcement of producer-consumer ordering rules. Specifically, interrupts issued by a device following a write operation must be processed only after the previous write operations have been completed and observed. Similarly, interrupts issued by a device must be observed before any subsequent read completions generated by the device.

MSI is the preferred mechanism for interrupt signaling in PCIe due to its efficiency and support for low-latency communication between devices and harts. By adopting MSI, systems can achieve faster and more reliable interrupt handling, essential for high-performance computing environments.

IIC_030

The Incoming Message-signaled Interrupt Controller (IMSIC) MUST implement an interrupt file for S-mode.

IIC_040

The IMSIC MUST support at least 5 VS-mode interrupt files.

Supporting 5 VS-mode interrupt files for a hart allows context switching between up to 5 virtual CPUs (vCPU) on a hart without needing to swap the contents of the interrupt file out to memory. This is particularly beneficial when devices are directly assigned to virtual machines (VMs), as swapping out the context of an IMSIC interrupt file may result in longer latencies due to the need to redirect device interrupts to a memory-resident interrupt file.

IIC_050

The S-mode interrupt file MUST support at least 255 interrupt identities.

IIC_060

The VS-mode interrupt files MUST support at least 63 interrupt identities.

IIC_070

IIC_080

SoC devices using wire-signaled interrupts must implement the rules related to ordering of interrupts vs. older read/writes from devices as specified by the device and/or bus interface specifications that such devices conform to. See also SID_010.

IOM_010

All IOMMUs in the SoC MUST support the RISC-V IOMMU specification [9].

IOM_020

DMA capable peripherals being governed by an IOMMU allows OS/hypervisors to restrict DMA originating from such devices to a subset of memory to enhance security and software fault tolerance. The address translation capability provided by the IOMMU enables usages such as passthrough of such devices to virtual machines, shared virtual addressing, etc.

The number of IOMMUs implemented in the SoC to satisfy rule IOM_020 is UNSPECIFIED.

IOM_030

The IOMMU governing a PCIe root port MUST support at least 16-bit wide device IDs.

IOM_040

An IOMMU that does not govern a PCIe root port MUST support a device ID width required to support all requester IDs originated by the devices governed by that IOMMU.

IOM_050

The IOMMU MUST implement all the page based virtual memory system modes and extensions that are implemented by the RISC-V application processor harts in the SoC.

The page based virtual memory system modes supported by the IOMMU are enumerated in the IOMMU capabilities register.

IOM_070

The IOMMU SHOULD support pass-through mode and MRIF mode MSI address translation.

IOM_080

When MRIF mode MSI address translation is supported, the IOMMU MUST support atomic updates to the MRIF (enumerated by 1 setting of capabilities.AMO_MRIF).

IOM_090

IOMMU governing PCIe root ports SHOULD support PCIe address translation services (ATS).

High performance devices such as DPU/SmartNICs, GPUs, and FPGAs, utilized in server platforms rely on ATS and Page Request services to achieve high throughput and low-latency I/O. Supporting ATS is also required for efficiently accommodating usage models such as Shared Virtual Addressing and direct work submission from user mode.

IOM_100

IOMMU governing PCIe root ports SHOULD support the T2GPA mode of operation with ATS if ATS is supported.

The T2GPA control enables a hypervisor to prevent DMA from a device, even if the device misuses the ATS capability and attempts to access memory that is not explicitly authorized by the page tables governing that device’s memory accesses. The threat model could also include a man-in-the-middle on the PCIe link inserting ATS-translated requests to access memory that was not previously authorized. As an alternative to setting T2GPA to 1, the hypervisor might establish a trust relationship with the device if authentication protocols such as SPDM are supported by the device. For PCIe, for example, the PCIe Component Measurement and Authentication (CMA) capability provides a mechanism to verify the device’s configuration and firmware/executable (Measurement) and hardware identities (Authentication). This mechanism establishes such a trust relationship, and the PCIe link may be integrity-protected using PCIe integrity and data encryption (IDE) to defend against a man-in-the-middle adversary.

IOM_110

IOMMU governing RCiEP MUST support PCIe address translation services (ATS) if any of the RCiEPs governed by the IOMMU support the ATS capability.

IOM_120

IOMMU governing RCiEP MAY support the T2GPA mode of operation with ATS if ATS is supported.

The threats associated with misuse of ATS or malicious insertion of ATS translated requests by a man-in-the-middle may not be present with RCiEP being integrated in the SoC.

IOM_130

IOMMU MUST support MSI and MAY support wire-signaled interrupts for external interrupts originated by the IOMMU itself.

IOM_140

IOMMU MUST support little-endian memory access to its in-memory data structures.

IOM_150

IOMMU MAY support big-endian mode memory access to its in-memory data structures.

The IOMMU memory-mapped registers always have a little-endian byte order.

IOM_160

IOMMU MAY support the PCIe PASID capability.

IOM_170

IOMMU that supports PASID capability MUST support 20-bit PASID width and MAY support 8-bit and 17-bit PASID widths.

PCIe specification strongly recommends that hardware implement the maximum width of 20 bits to ensure interoperability with system software. See also the implementation note on PASID width homogeneity in the PCIe specification 6.0 section 6.20.2.2.

IOM_180

IOMMU SHOULD support a hardware performance monitor (HPM).

The HPM is a valuable tool for system integrators for performance monitoring and optimizations. An IOMMU is highly recommended to provide an HPM.

IOM_190

An IOMMU that supports an HPM MUST support the cycles counter.

IOM_200

An IOMMU that supports an HPM MUST incorporate at least 4 event counters.

A typical performance analysis operation may involve simultaneously counting the number of translation requests, IOATC misses, and page table walks. An HPM with sufficient number of event counters ensures accurate and comprehensive data collection, enabling detailed performance analysis and optimization.

IOM_210

The cycles counter and the event counters MUST be at least 40 bits wide.

IOM_220

The IOMMU SHOULD support the software debug capabilities enumerated by DBG field in the capabilities register.

IOM_230

The physical address width supported by the IOMMU MUST be greater than or equal to the physical address width supported by the RISC-V application processor harts in the SoC.

Having the physical address width greater than or equal to the width supported the harts in the SoC enables use of all addressable memory for I/O and facilitates the sharing of page tables between the hart MMU and the IOMMU.

IOM_240

The reset default of the iommu_mode MUST be Off.

The IOMMU disallowing DMA unconditionally following reset due to the mode being Off allows the SoC firmware and software to enable DMA when suitable security protections as required have been established. The IOMMU mode being Off at reset does not pose a significant issue to SoC firmware that needs to employ DMA (e.g., for firmware loading) as that firmware may program the mode in the appropriate IOMMU prior to programming the peripheral governed by that IOMMU to perform a DMA.

IOM_250

An IOMMU that is implemented as an RCiEP MUST use base class 08H and subclass 06H [10].

The base class 08H and sub-class 06H are designated by PCIe for use by an IOMMU. Implementing the IOMMU as a PCIe device allows an operating system to determine a driver for the IOMMU and to assign resources such as interrupt vectors to the IOMMU in a PCIe compatible manner.

IOM_260

The host bridge MUST enforce the physical memory attribute checks and physical memory protection checks on memory accesses originated by the IOMMU and signal detected access violations to the IOMMU.

These checks are analogous to the PMA and PMP checks performed by the RISC-V hart. The host bridge (also known as IO bridge) invokes the IOMMU for address translations. To perform the operations requested by the host bridge the IOMMU may need to access in-memory data structures such as the device directory table and page tables.

The physical memory protection limit access from IOMMUs to phusical addresses to support secure processing and contain faults. These checks allow restricting the IOMMU to only have access to the same memory that the hart software that programs the IOMMU has access to.

The IOMMU specification requires an IOMMU to support locating IOMMUs in-memory data structures, in-memory queues, and page tables in memory address ranges that hold main memory. Support for locating these in I/O memory is not required.

IOM_270

An IOMMU MUST support 24-bit device IDs if the IOMMU governs multiple PCIe root ports that may be part of different PCIe hierarchies.

An IOMMU governing PCIe root ports uses requester ID (RID) - the tuple of bus/device/function numbers (or just bus/function numbers, if the PCIe ARI option is used) - to locate a device context to use for address translation and protection. The 16-bit RID uniquely identifies a requester within a hierarchy. This RID needs to be augmented with the Hierarchy ID (also known as segment ID) - an 8-bit number - to uniquely identify a requester across PCIe hierarchies.

IOM_280

The host bridge MUST provide the PCIe RID as the bits 15:0 of the device_id input to the IOMMU for requests from PCIe EPs and RCiEP.

IOM_290

When the IOMMU supports 24-bit device IDs, the host bridge MUST specify the segment number associated with the PCIe hierarchy from which requests were received as the bits 23:16 of the device_id to the IOMMU.

IOM_300

The determination of device_id input to an IOMMU for requests originating from non-PCIe devices is UNSPECIFIED. If PCIe and non-PCIe endpoints/RCiEP are governed by the same IOMMU, the SoC MUST ensure that there is no overlap between any device_id associated with non-PCIe devices with any device_id formed using the PCIe RID (and if applicable the segment ID).

IOM_310

The host bridge MUST provide the 20-bit PASID from the PCIe PASID TLP Prefix as the process_id input to the IOMMU along with an indication about the validity of the process_id input. When the process_id is indicated as valid, the host bridge MUST additionally provide the "Execute Requested" and the "Privilege Mode Requested" bits from the PASID TLP prefix as input to the IOMMU. When process_id input is indicated as not valid, the host bridge MUST set the "Execute Requested" and "Privilege Mode Requested" inputs to 0.

The host bridge providing the full 20-bit value without truncation from the PASID TLP prefix to the IOMMU enables the IOMMU to determine if the PASID value is wider than supported by the current configuration of the process directory table for that device and generate a fault notification if so.

IOM_320

The determination of process_id, "Execute Requested", and "Privilege Mode Requested" inputs to an IOMMU for requests originating from non-PCIe devices is UNSPECIFIED.

RAS_010

The level of RAS implemented by the SoC is UNSPECIFIED.

The level of RAS implemented by an SoC depends on the reliability goals established for the SoC, which are commonly measured using metrics such as failure-in-time (FIT) and defects-per-million (DPM). Achieving these goals requires a combination of fault prevention, error detection, and error correction techniques.

This specification strongly recommends the implementation of error detection and correction codes for storage elements like significant caches and memories. Furthermore, it suggests utilizing mechanisms such as single-symbol (SSC) ECC in DRAM controllers to address failure scenarios, such as when all bits in a single DRAM device experience a failure.

Additionally, this specification encourages the adoption of mechanisms like periodic scrubbing, also known as patrol scrubbing. These mechanisms proactively identify and rectify errors before they accumulate to a critical point, surpassing the capability of the implemented error correction codes. For instance, this could involve addressing situations where single bit errors escalate into double bit errors, surpassing the correction code’s capacity.

RAS_020

SoC SHOULD support the generation, storage, and forwarding of poisoned data. The granularity at which data is poisoned is UNSPECIFIED.

When an uncorrected data error is detected by a component, it might allow potentially corrupted data to reach the data requester, but with an associated poison indicator. These errors are referred to as uncorrected deferred errors (UDE), as they enable the detecting component to continue functioning and postpone addressing the error until a later time, assuming the poisoned data gets consumed. If a component (such as a hart, an IOMMU, a device, etc.) consumes the poisoned data, it triggers an uncorrected urgent error (UUE), leading to the invocation of a recovery handler for immediate remedial actions, as further deferral of the error is not feasible.

The technique of data poisoning facilitates delaying the handling of uncorrected errors until the moment the corrupted data is actually consumed. Data poisoning offers a more precise identification of the software and/or hardware component affected by the data corruption. This specificity allows for targeted recovery actions that impact only the affected components.

To ensure the integrity of the poisoned data indicator when stored, error detection and correction codes should be applied. This practice prevents subsequent errors from leading to the silent consumption of the corrupted data.

Data poisoning also empowers the implementation of error containment features supported by industry standards like PCIe and CXL.

For more detailed discussions on the treatment of faults and errors, refer to the RISC-V RERI specification.

RAS_030

If poisoned data needs to be transmitted from a first component to a second component that lacks the ability to manage poison, the first component MUST trigger an critical uncorrected error report instead of silently transmitting the corrupted data.

Some components serve as intermediaries through which data passes. For instance, a PCIe/CXL port acts as an intermediary that receives data from memory but doesn’t consume it; rather, it forwards the data to an endpoint. In such cases, the intermediary component might encounter poisoned data. While this component can propagate the error and avoid logging an error, a different scenario arises when the destination component (such as a PCIe endpoint) cannot handle poison. In such situations, the originating component must trigger an urgent error signal instead of transmitting the poisoned data without the associated poison indicator. Failing to do so would breach the containment of the corrupted data during propagation.

RAS_040

The SoC SHOULD support the RISC-V RAS error record register interface (RERI) [11] for error logging and signaling.

RAS_050

When RERI is supported, the RAS error records MUST include the capability to individually enable error signaling for each severity - Uncorrected Error Critical (UEC), Uncorrected Error Deferred (UED), and Corrected Error (CE) - of error that could be logged in that specific error record.

Configurable enables provide software with the flexibility of using an event-based or polling-based error logging for both corrected errors and deferred errors. Typically, software operates in an event-based mode for critical errors, as these errors necessiate immediate remedial action when they arise.

RAS_060

If RERI is supported, RAS error records MUST preserve the state of logged error information (including status, address, information, supplemental information, and timestamp) across a RAS-initiated reset. The state of RAS error records MAY persist across other types of implementation-defined resets. After a reset, including a RAS-initiated reset, the state of the control register in the RAS error record is considered UNSPECIFIED.

Some errors may lead a hardware component to enter a failure mode in which it becomes incapable of servicing additional requests- colloquially termed 'jammed' or 'wedged'. In these situations, the SoC may require a reset to restore it to an operational state (a RAS-initiated reset). Preserving the RAS error records through such resets enables the SoC firmware and system software to retrieve these error records during boot following such a reset, facilitating logging and analysis.

RAS_070

If RERI is supported, the RAS error records MAY support error record injection, which is intended to facilitate RAS handler verification.

Verifying the correct implementation of RAS handlers presents a formidable challenge, given the impracticality of deterministically inducing all potential errors within the SoC to validate the RAS handler’s adherence to desired recovery protocols. An unverified RAS handler can lead to undesired behavior during error occurrences, potentially reducing SoC availability or affecting its serviceability.

To address this, error record injection offers a convenient method for conducting such verification. It allows the introduction of a range of error signatures, which can then be signaled and observed. While hardware error injection techniques also offer a means of verification (e.g., methods to intentionally corrupt a data location protected by an error detection code), providing open access to these capabilities for software use might not align with security and stability concerns.

RAS_080

If RERI is supported, then the hardware components in the SoC that support error correction MUST incorporate a corrected error counter within their respective error records. Additionally, these components MUST support the signaling of counter overflows.

Counting corrected errors offers a more precise assessment of system reliability. Enabling signaling upon counter overflow empowers software to define a suitable threshold for logging and analysis of these corrected errors.

Certain hardware units might maintain a history of corrected errors and increment the corrected error counter only if the error differs from a previously reported one. Additionally, some hardware units could incorporate low-pass filters like leaky buckets, which regulate the rate at which corrected errors are reported and counted. This rule pertains to corrected errors tracked by the error record once the hardware component determines reporting and counting based on its specific filtering rules.

QOS_010

The SoC SHOULD incorporate QoS mechanisms to mitigate unwarranted performance interference that arises when multiple workloads access shared resources like caches and system memory.

QOS_020

The SoC SHOULD integrate support for the RISC-V capacity and bandwidth controller register interface (CBQRI) [12] in significant shared caches and the memory controllers.

QOS_030

If CBQRI is supported, RISC-V harts within the application processors of the SoC MUST include support for the srmcfg CSR. Furthermore, this CSR MUST support a minimum of 16 RCIDs and at least 32 MCIDs. The count of RCID and MCID that can be used in the SoC SHOULD scale with the number of RISC-V harts in the SoC.

The srmcfg CSR is provided by the Ssqosid extension [13].

QOS_040

If CBQRI is supported, the IOMMUs in the SoC SHOULD incorporate support for the CBQRI-defined extension, enabling the association of RCID and MCID with requests initiated by devices and the IOMMU.

QOS_050

If CBQRI is supported, significant caches such as the last-level cache in the SoC SHOULD support cache capacity allocation.

QOS_060

If CBQRI is supported, significant caches such as the last-level cache in the SoC SHOULD incorporate support for monitoring cache capacity usage.

QOS_070

If CBQRI is supported, the memory controllers within the SoC SHOULD include support for bandwidth allocation.

QOS_080

If CBQRI is supported, the memory controllers in the SoC SHOULD include support for monitoring bandwidth usage.

The method employed by the SoC for bandwidth throttling and control is specific to its implementation. It is advisable for the implementation to utilize a scheme that results in a deviation of no more than +/- 10 % from the target set by system software through the CBQRI interface.

QOS_090

If CBQRI is supported, the count of RCID and MCID supported by capacity controllers, bandwidth controllers, and all RISC-V application processor harts in the SoC MUST be consistent.

Portable system software could opt to limit itself to accommodating the minimum count of RCID and MCID across the controllers. This approach avoids the complexity of dealing with unequal numbers of RCID and MCID across controllers, which would otherwise necessitate intricate allocations and constraints on workload placement.

QOS_100

If CBQRI is supported, the monitoring counters in the capacity and bandwidth controllers MUST be sufficiently wide to not overflow when sampled at a rate of 1 Hz.

As an illustration, consider an HBM3 memory interface that can facilitate data transfers at a rate of up to 1 TB/s. This scenario would necessitate a 34-bit counter to prevent overflow when sampled at a frequency of 1 Hz.

MNG_010

The SoC SHOULD incorporate support for an x1 PCIe lane, preferably Gen 5, but at least Gen 3, to establish a connection with the BMC.

This interface is commonly linked to a BMC as a PCIe endpoint, serving various purposes. These include facilitating host-to-BMC communication for tasks like video output (e.g., remote KVM support), MCTP transport over PCIe VDM, and hosting a USB controller. The BMC might also support remote presence capabilities, like remote media redirection and support for keyboard and mouse functions through virtual USB.

The in-band network interfaces serve as communication channels for system software to interact with the BMC. This interaction employs protocols like the Redfish host interface.

Furthermore, the PCIe interface to the BMC empowers the BMC, using SoC-routed PCIe VDMs, to utilize these VDMs for transmitting MCTP messages. These messages manage platform devices, including network controllers, NVMe controllers, FPGAs, GPUs, and more.

MNG_020

MNG_030

SPM_010

It is recommended that a cache with a capacity that is approximately 16 KiB or larger be considered a significant cache.

SPM_020

SPM_030

SPM_040

Bandwidth and latency are the most commonly used performance metrics to guide workload placement and tuning.

SPM_050

SPM_060

All PCIe Gen6 ports within the SoC SHOULD incorporate support for the Flit performance measurement extended capability defined by PCIe specification 6.0.

SEC_010

A root of trust (RoT) is the foundation on which all secure operations of a system depend. A hardware RoT is a dedicated and possibly isolated trusted subsystem that can provide stronger protections against physical and logical attacks.

SEC_020

The PCIe root ports within the SoC SHOULD support PCIe Integrity and Data Encryption (IDE) capability.

The IDE extension adds optional capabilities to perform hardware encryption and integrity checks on packets transferred across PCIe links. This addition provides confidentiality, integrity, and replay protection against hardware-level attacks.

SEC_030

The SoC SHOULD support encryption of off-chip DRAM using a transient memory encryption key that has at least 256-bit key lengths.

Off-chip memory encryption provides protection to critical assets in memory such as credentials, data encryption keys, and other secrets.

SEC_040

The cryptographic modules used to implement PCIe and off-chip DRAM encryption SHOULD comply with security requirements specified by relevant security standards from national standards laboratories.

FIPS 140-3 is an example of such a standard

SEC_050

The SoC SHOULD have the capability of interfacing with a Trusted Platform Module (TPM) that adheres to the TPM 2.0 Library specification [21].

A TPM enhances security by providing secure storage for sensitive information such as credentials and passwords, cryptographic operations and protection against tampering or unauthorized access.