Myri-10G 10-Gigabit Ethernet Solutions

Myri-10G 10-Gigabit Ethernet Solutions
for Linux, Windows, Solaris, Mac OS X, FreeBSD, and VMware ESX

For standard TCP/IP and UDP/IP 10-Gigabit Ethernet applications, Myri-10G Network Interface Cards (NICs) and software deliver excellent performance at low cost, and are fully compliant with Ethernet standards.

Performance. The standard netperf benchmark results below are with Linux 2.6.20 between servers with two Intel quad-core 2.66GHz Xeon X5355s:

Netperf Test   MTU    BW        TX_CPU %  RX_CPU %
------------   ----   -------   --------  --------
TCP_STREAM     9000   9910.26    11.32      5.89
TCP_SENDFILE   9000   9894.46     3.37      5.91
TCP_STREAM     1500   9463.06    10.54      8.71
TCP_SENDFILE   1500   9354.66     2.75      8.67

Thanks to the offloads in the driver and NIC firmware, the host-CPU load is quite low even at these near-wire-speed throughput rates. Another factor that is important to the near-wire-speed performance of Myri-10G NICs is that they connect to hosts through PCI Express x8, a 16+16 Gigabit/s full-duplex I/O fabric that is fast enough, even after PCI Express protocol overhead, to keep up with the 10+10 Gigabit/s full-duplex network port. See this Myri-10G 10-Gigabit Ethernet Performance Measurements page for additional benchmarks, including with other operating systems.

Preferred NICs for 10-Gigabit Ethernet Applications. Myri-10G NICs are available in the standard PCI Express add-in card and other form factors, and with a full choice of PHYs for the network ports. For complete details, see the Myri-10G Product List and this Guide to Myri-10G PHYs (pdf). However, the most popular NICs for 10-Gigabit Ethernet applications are these:

*Selection Guide for 10-Gigabit Ethernet NICs*
Network Port	SFP+ transceiver socket for 10GBase-SR or 10GBase-LR over serial fiber, or SFP+-terminated twinax or other cables	CX4 connector for 10GBase-CX4 over copper cables
NICs with one network port	10G-PCIE-8A-S NIC Low-profile (half-height) PCI Express add-in card (Standard PCI faceplate version shown) $695 list price (SFP+ transceiver sold separately)	10G-PCIE-8AL-C NIC Low-profile (half-height) PCI Express add-in card (Low-profile PCI faceplate version shown) $595 list price
NICs with two network ports for failover	10G-PCIE-8B-2S NIC Low-profile (half-height) PCI Express add-in card (Standard PCI faceplate version shown) $795 list price (SFP+ transceivers sold separately)	To be introduced soon 10G-PCIE-8B-2C NIC
Cables	LC-connectorized duplex multimode fiber cables for use with 10GBase-SR SFP+ transceivers, or LC-connectorized single-mode fiber cables with 10GBase-LR SFP+ transceivers, or SFP+-terminated twinax or other cables	10GBase-CX4 copper cables

A note about SFP+: Myricom's original 10GBase-SR and 10GBase-LR NICs with 10G-PCIE-8A-R product codes use XFP fiber transceivers. SFP+ fiber transceivers, introduced more recently, are lower cost, lower power, and smaller size than XFP transceivers, so Myricom introduced the functionally equivalent 10G-PCIE-8A-S NICs. In addition, the SFP+ ports of Myri-10G NICs and switch line cards can be used with SFP+-terminated twinax copper cables (sometimes called "Direct Attach") and with other SFP+-terminated cables.

Software. The driver and NIC firmware for 10-Gigabit Ethernet operation is currently available for Linux, Windows, Solaris 10, Mac OS X, FreeBSD, and VMware ESX. You can download the drivers (which include the NIC firmware) from this Myri-10G 10-Gigabit Ethernet Driver Download page. The driver was contributed to and accepted in the Linux kernel, and is included in the 2.6.18 and later kernels and in many Linux distributions. However, the driver available by download will be more up-to-date. The driver is also included in FreeBSD 7.0.

Offloads. The driver and NIC firmware implement zero-copy on the send side with all supported operating systems, and, depending on the OS, use a variety of stateless offloads, including:

Interrupt Coalescing
IP and TCP checksum offload, send and receive
TSO (TCP Segmentation Offload, also known as Large Send Offload)
RSS (Receive-Side Scaling)
LRO (Large Receive Offload)
Multicast filtering

The driver and NIC firmware do not implement “stateful” offloads. These NICs and their software are deliberately not TCP Offload Engines. See, for example, http://linux-net.osdl.org/index.php/TOE for some of the reasons why TCP offload "has little merit."

10-Gigabit iSCSI. These NICs may also be used for 10-Gigabit iSCSI. See the Myri-10G iSCSI installation instructions and iSCSI performance measurements.

Form factors. All of the one-port and some of the two-port Myri-10G NICs are available as low-profile (half-height) PCI Express add-in cards. Low-profile cards allow the option of installation in 2U servers without the need for a riser card if you specify low-profile faceplates. NICs with -8A- or -8B- product codes have standard PCI faceplates. NICs with -8AL- or -8BL- product codes have low-profile PCI faceplates. Compare the PCI faceplates in the photos above to see the difference. Myricom also produces Myri-10G NICs in other form factors, including High Speed Expansion Cards (HSECs) for the IBM BladeCenter H. See the full Myri-10G product list for additional information.

Ethernet interoperability has been demonstrated with 10-Gigabit Ethernet switches from Myricom, Foundry, Extreme, Fujitsu, HP, Quadrics, SMC, Force10, Cisco, Blade Network, Broadcom, and Arista.

PCI Express interoperability has been verified with all of the common PCI Express chipsets and many commercially available motherboards. This tabulation of test results with PCI-Express motherboards shows their PCI Express performance, which can vary significantly between different products.

The power of programmable NICs. Myri-10G NICs include processors and firmware, and can be used in system solutions that go beyond the standard TCP/IP and UDP/IP over Ethernet. These same NICs can be used with the MX (Myrinet Express) software for kernel-bypass operation for low latency and very low host-CPU load over either Ethernet or Myrinet networks. See this Low-Latency 10-Gigabit Ethernet product brief (pdf). Other optional software distributions include Video Pump^TM for low-CPU-load UDP streaming for IPTV applications, and software for 10Gb Ethernet packet capture. Contact the Myricom Sales Team for additional information, which must generally be under an NDA.

It is also possible for customers to develop custom 10-Gigabit Ethernet drivers and firmware for demanding applications. Contact Myricom Tech Support for a discussion of your special requirements.

See also these product briefs:

Last updated: 2 December 2008