wire / wire-format / roce

RoCE (RDMA over Converged Ethernet) v2

RoCE v2 encapsulates InfiniBand transport (BTH) over UDP/IP/Ethernet. The packet: Ethernet header, IP header, UDP header (port 4791), then BTH, then payload. RoCE v2 uses standard IP routing, unlike RoCE v1 which uses Ethernet (not routable).

wire-format kind wire-format status standard verification verified tier B wire-format@1

aka: RoCE (RDMA over Converged Ethernet) v2

wire format

spec: RoCE (InfiniBand Trade Association)

RoCE v2 encapsulates InfiniBand transport (BTH) over UDP/IP/Ethernet. The packet: Ethernet header, IP header, UDP header (port 4791), then BTH, then payload. RoCE v2 uses standard IP routing, unlike RoCE v1 which uses Ethernet (not routable).

fieldsizemeaning
Ethernet header14 bytesDst MAC, Src MAC, EtherType 0x0800 (IPv4) or 0x86DD (IPv6).
IP header20 bytes (IPv4) or 40 bytes (IPv6)Standard IP header.
UDP header8 bytesSrc port (ephemeral), Dst port 4791, length, checksum.
BTH12 bytesInfiniBand Base Transport Header.
PayloadvariableRDMA operation payload.
ICRC4 bytesInvariant CRC (over BTH + payload).
FCS4 bytesFrame Check Sequence (Ethernet CRC).

example:

ff ff ff ff ff ff 00 00 00 00 00 00 08 00

RoCE v2: Ethernet header with EtherType 0x0800 (IPv4)


provenance

agent: curl -H 'accept: application/json' wire.phall.io/wire-format/roce or /wire-format/roce.json