MX-2G 1.0 Performance Measurements
MX-2G 1.0 Performance Measurements
MX 1.0 API Performance
with M3F-PCIXD, M3F2-PCIXE, and
M3F-PCIXF Myrinet-2000/PCI-X NICs
June 2005
The following tables summarize performance measurements for M3F-PCIXD, M3F2-PCIXE and M3F-PCIXF Myrinet NICs with MX-2G 1.0.0 in dual 1.8GHz Opteron hosts with the AMD-8131 chipset. These hosts have 64-bit, 133MHz, PCI-X slots with good PCI-DMA performance (824 MB/s bus read, 1032 MB/s bus write, according to the MX "mx_dmabench" utility).
| Performance Metric | M3F-PCIXD 64-bit, 133MHz PCI-X 225MHz RISC & Memory |
| Sustained one-way data rate for large messages | 247 MByte/s |
| Sustained two-way data rate for large messages | 494 MByte/s |
| Latency for null messages | 3.2 µs (MPI 3.2 µs) |
| Performance Metric | M3F2-PCIXE 64-bit, 133MHz PCI-X 333MHz RISC & Memory |
| Sustained one-way data rate for large messages | 495 MByte/s |
| Sustained two-way data rate for large messages | 912 MByte/s |
| Latency for null messages | 2.6 µs (MPI 2.7 µs) |
| Performance Metric | M3F-PCIXF 64-bit, 133MHz PCI-X 333MHz RISC & Memory |
| Sustained one-way data rate for large messages | 248 MByte/s |
| Sustained two-way data rate for large messages | 495 MByte/s |
| Latency for null messages | 2.5 µs (MPI 2.6 µs) |
These results are representative of message-passing performance in application programs. They are not "marketing benchmarks". The performance measurements are between user processes, with full protection (safe in multi-user, multiprogramming environments), and with end-to-end data-integrity checking. Other host computers may show better or worse performance. However, because the operating system is bypassed for the MX API, there is little performance variation between different operating systems.
Sustained one-way data rate
As shown in the following graph, the sustained (streamed), one-way, data rate for MX closely approaches the unidirectional data rate of a Myrinet link for long messages (250 MB/s for M3F-PCIXD and M3F-PCIXF, 500 MB/s for M3F2-PCIXE). The data rate reaches half of the asymptotic value with message lengths of ~450B for M3F-PCIXD, ~650B for M3F2-PCIXE and ~300B for M3F-PCIXF NICs.
The test was performed by one host sending and another host receiving messages of each plotted length repeatedly.
Last updated: 02 June 2006