RDMA (Remote Direct Memory Access) is a network communication technology derived from DMA (Direct Memory Access). It enables user applications to bypass the operating system kernel (CPU) and directly interact with network interface cards (NICs) for data transmission, delivering ultra-high bandwidth and ultra-low latency. Initially developed for high-performance computing (HPC), RDMA has now become widely adopted in data centers。

1. RDMA Technical Principles

The core principle of RDMA lies in leveraging specialized hardware on network adapters (e.g., RDMA engines on NICs) to bypass the operating system kernel and perform direct memory transfers below the network transport layer. This mechanism minimizes latency and CPU overhead while enhancing network efficiency。

In RDMA, the communication between the sender and the receiver is carried out through the RDMA adapter. First, the sender registers the data buffer to be sent in the memory of the RDMA adapter and sends its descriptor (containing the address, size, and other information of the data) to the receiver. The receiver registers the receive buffer with the RDMA adapter and informs the adapter of the address of the receive buffer. Then, the sender transfers the data directly from the send buffer to the receive buffer through the RDMA adapter without the intervention of the operating system. This process is called an RDMA write operation. The receiver can also read data directly from the sender's memory through an RDMA read operation.

RDMA supports three basic types of operations: write operations, read operations, and atomic operations. Write operations allow a node to write data directly to the memory of another node. Read operations allow a node to directly read data from the memory of another node. Atomic operations are special operations that can perform some atomic operations on remote nodes, such as compare-and-swap (CAS).

The working principle of RDMA is mainly to reduce the delay of data transmission and the load of CPU through hardware-level direct memory access. Compared with the traditional TCP/IP communication method, RDMA does not need to establish a connection between the sender and the receiver, nor does it need to perform complex protocol processing, so it has higher performance and lower latency. In addition, RDMA also supports features such as zero copy and kernel bypass, which can further improve the efficiency of data transmission.

In general, the technical principle of RDMA is to use dedicated hardware on the network adapter through direct memory access, bypass the operating system kernel, and achieve efficient, low-latency, and low-CPU occupancy network communication.

2. The technical characteristics of RDMA mainly include the following aspects:

* No CPU intervention required: One of the core characteristics of RDMA technology is that it does not require CPU intervention. Applications can directly access the remote host memory without consuming any CPU resources in the remote host. This greatly reduces the delay and CPU occupancy during data transmission and improves overall performance.

* Kernel bypass: RDMA supports kernel bypass technology, that is, applications can directly perform data transmission in user mode without context switching between kernel mode and user mode. This further reduces the delay of data transmission and improves efficiency.

* Zero copy: RDMA technology has the characteristic of zero copy, which avoids the process of CPU copying data from one storage to another. In TCP/IP communication, data transmission between hosts requires frequent copy operations, and RDMA reduces these unnecessary copy operations through direct memory access, thereby improving the efficiency of data transmission.

* High bandwidth and low latency: Since RDMA bypasses the protocol stack provided by the operating system and directly performs data transmission under the network transport layer, it has higher bandwidth utilization and lower latency. This makes RDMA very suitable for network communication scenarios that require high performance and low latency.

* Hardware support: RDMA requires specific network card hardware support, which usually has a dedicated RDMA engine for processing RDMA communication. This hardware support enables RDMA to achieve its high performance and low latency characteristics.

In general, the technical features of RDMA include no CPU intervention, kernel bypass, zero copy, high bandwidth and low latency, and hardware support. These features make RDMA an efficient and low-latency network communication technology, especially suitable for scenarios that require processing large amounts of data and high-performance network communications.

3. Application Scenarios of RDMA

The application scenarios of RDMA technology are very wide. The following are some of the main application scenarios:

* Data Center: In a data center, servers need to frequently transmit and exchange large amounts of data. RDMA's high performance and low latency make it very suitable for network communications in data centers, which can improve data transmission efficiency, reduce latency, and improve overall performance.

*  High Performance Computing (HPC): The field of high performance computing requires processing large amounts of data and performing complex calculations. RDMA's direct memory access and zero copy technology can reduce data transmission overhead and improve computing efficiency. At the same time, RDMA supports large-scale parallel processing and is very suitable for multi-node parallel computing in HPC environments.

* Distributed Storage System: In a distributed storage system, nodes need to frequently perform data read and write operations. RDMA's efficient data transmission and direct memory access characteristics can improve the data read and write efficiency between nodes and improve the overall performance of distributed storage systems.

* Cloud computing: In a cloud computing environment, communication and data transmission between virtual machines (VMs) are common requirements. RDMA can be used to achieve efficient communication between VMs and improve the overall performance of the cloud computing environment.

In general, although RDMA has some shortcomings and limitations, it is still a very effective network communication technology in many high-performance, low-latency scenarios. When choosing whether to use RDMA, you need to evaluate its applicability and performance based on specific application requirements and environments.