CSR Labs - High Availability with home hardware

Raspberry Pi CM4 Dual Ethernet Card

Waveshare CM4 dual ethernet mini

Despite its small size and limited resources if compared to a full featured computer, a Raspberry Pi can indeed act as a real server.

One of the key benefits of using a Raspberry Pi as a server is its low cost and small form factor.
This makes it an ideal choice for projects that require a low-power, low-cost server, such as home automation, home media servers, or small-scale web servers.

Even with such a tiny machine we can build a highly available, fault tolerant system.

In order to achive that, one of the main starting points is having a dual NIC connection, which provides several benefits:

  • Redundancy: If one Ethernet connection fails, the other can take over, ensuring that the system remains connected and able to communicate.
  • Improved reliability: With two Ethernet connections, the system can continue to operate even if one connection is experiencing issues.
  • Maintenance: On dual NIC systems, being able to disconnect one cable at a time can make maintenance easier and less disruptive, allowing you to make changes to the system without affecting network traffic.

Overall, a dual Ethernet connection can provide a more robust and reliable network connection, helping to ensure that the system remains available and responsive even in the event of a failure.

There are a few hardware choices to provide a Raspberry Pi with two Ethernet connections.
This great article by Jeff Jeerling goes through several aspects related to having a dual NIC card on a Raspberry Pi system and focuses on the benefits of a NIC attached to a PCI Express lane rather than a USB-to-Ethernet controller.

I tested two Ethernet-to-SPI bus cards, which did the job, but both of them had issues that I could not completely resolve.

 

ENC28J60 Ethernet to SPI module

ENC28J60 Ethernet to SPI card

These modules are usually targeted to Arduino users but can be used with Raspberry Pi too.
The Raspberry Pi OS provides an overlay for ENC28J60 based cards that can be easily configured and seen by the operating system as a second NIC. However these cards do not support duplex autonegotiation, and they will be detected as a half-duplex device. To use full duplex both ends must be confiured manually, meaning that a managed switch is required, togheter with some additional configuration on the Raspberry Pi side too. In addition the speed is limited to 10 Mb/s.

Below is a screenshot taken on a Raspberry Pi 3 with a ENC28J60 card attached to the SPI bus and configured with ifenslave.

ENC28J60 bonding on Raspberry Pi 3

 

WIZnet W5500 Ethernet to SPI module

WIZnet W5500 Ethernet to SPI card

The WIZnet W5500 module is an Ethernet to SPI adapter with a theoretical speed of 100 Mb/s, but in practice such a speed is never achieved on a Raspberry Pi due to the SPI bus frequency limit.
I tested this module on a Raspberry Pi 4 (pic below).

WIZnet W5500 with Raspberry Pi 4

The major issue with this module is that Raspberry Pi OS overlay is missing support for the PHY status, resulting in the OS being unable to know the actual interface status, see the pic below.

W5500 unknown status

This has been discussed in this thread of the official Raspberry Pi forum. At the time of writing, I could not find a way to fix it.

 

Waveshare Mini Dual Gigabit Ethernet Base Board

Waveshare Mini Dual Gigabit Ethernet Base Board

The Waveshare Mini Dual Gigabit Ethernet Base Board is a compact card designed for the Raspberry Pi Compute Module 4, featuring dual Ethernet RJ45 ports. The second NIC is achieved through a Realtek RTL8111H chip, a 10/100/1000M Ethernet to PCI Express controller, which combines a triple-speed IEEE 802.3 compatible Media Access Controller (MAC) with a triple-speed Ethernet transceiver, a PCI Express bus controller, and embedded memory.

This tiny card exposes the classical 40-pin GPIO and a USB 2.0 connector, but it is missing HDMI and audio. However, this is not a big issue for use as a headless server.

There is also a 4-pin FAN connector that uses 2 GPIO ports, as we can see in the schematic below.

Fan connector schematic

The key feature of this card is its second ethernet controller, which is connected to a PCI express lane, while most of the similar cards use a USB-to-Ethernet controller. The benefits of this design are faster data transfer speeds, lower latency, more efficient use of system resources, and better compatibility.

In my tests, this card performed very well, so it has become the hardware of my choice to implement my home High Availability system.

Here are some screenshots of the card detected by ethtool and the configuration with the kernel bonding module.

Dual NIC card ethtool

Dual NIC card ethtool

In this guide, I went through all the steps to configure this dual NIC card to work as a fault tolerant Ethernet.

Support this blog


Please consider making a donation if you liked my blog and found it useful for your project.
It will support me in producing fresh articles and conducting more hardware testing.

PcbWay partnership program

Social

  1. GitHub
  2. Twitter