Hyperconverged Supermicro a2sdi-4c-hln4f

This howto aims at describing the choices and the build of a compact homelab with a hyperconverged chassis based on a Supermicro A2SDi-4C-HLN4F motherboard. The hypervisor OS will be Slackware64-current (with Qemu/KVM for virtualization), the storage will be provided by a Truenas core VM (thanks to pci-passthrough) and network orchestrated by an OPNSense VM.

Motivations

Why such a motherboard with a modest 4 cores Intel Atom C3558 ? Let's see the advantages :

• obviously, it's very compact (mini-itx form factor)
• up to 256 GB ECC RDIMM RAM supported
• the CPU has a very low TDP (~ 17W), so no need for a fancy and potentially noisy cooling solution
• 4 * 1 Gb/s Ethernet ports (cool for a network appliance such as OPNSense and there's no need for 10 Gb/s for this project)
• dedicated IPMI Ethernet port
• the SATA ports are provided by two distinct PCIe lines (see below, very important for pci-passthrough and no need for an additionnal HBA card)

A2SDi system block diagram

Well, the system has drawbacks too :

• the CPU power will not be extraordinary
• it's not possible to put a fan directly on top of the CPU heatsink (more on that later)
• no external USB 3.0 ports (ony one "internal", on the motherboard itself) ; this is desirable to plug an external HDD for local backups (USB 2.0 is too slow)

Hardware

2.5 or 3.5 inches drives ?

What is required for this project :

• mini tower format chassis
• Hypervisor OS on mirror (RAID 1 in zfs terminology) with two drives
• at least 4 TB of encrypted data with a minimum of resilience (mirror or raidz1 / raidz2 ...)
• it is recommended not to exceed 80% of the capacity of a ZFS filesystem, so, for 4 TB of usable data, 5 TB of raw capacity is required
• all disks easily accessible from the front of the chassis

Choosing between 2.5 (SFF) or 3.5 (LFF) inches drives is not so easy. Obviously, for raw capacity over a gigabit network, LFF HDD are unbeatable in terms of price. In France (as of march 2021), it costs ~ 80 euros for a 2 TB NAS 3.5 LFF HDD ... Same price for a 1 TB NAS SFF HDD. Moreover, SFF HDD over 2 TB simply doesn't exist for consumer NAS systems (enterprise SFF HDDs exist but are way too expensive). LFF HDD are a clear winner ? What about chassis size, cooling, noise and iops ?

At this point, there are two choices :

• 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 2 * 6 TB LFF HDD (mirror storage)
• 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 4 * 2 TB SFF SSD (raidz1 storage)

The most reasonable would be to use LFF HDDs but silent operation in a tiny chassis with decent iops are very important, so let's stick with the "unreasonable" configuration !

Cooling the CPU

As seen earlier, a standard CPU fan cannot be fitted on the heat sink.

Final system

• Chieftec mini tower case
• Be Quiet 400W 80plus gold PSU
• Supermicro A2SDi-4C-HLN4F
• 64 GB DDR4 RAM ECC (4 * 16 GB RDIMM)
• ToughArmor MB992SK-B for hypervisor and VMs OS with 2 * WD SA500
• ToughArmor MB994SP-4SB-1 for data storage (with fans off) with 4 * Crucial MX500
• PCIe card with 2 * USB 3.0 ports

• 

  Inside the chassis

• 

  Details

Installation

BMC setup

First things first, taking control of the BMC. These days, Supermicro's BMC are configured by default for DHCP and dedicated Ethernet port. There is no more the "ADMIN" default password but a random one written on a sticker directly on the motherboard (along with the MAC address of the BMC). Be sure to remember these informations before plugging the dedicated port on a switch of your your LAN.

With the MAC address in mind, it should be easy to retrieve the IP address assigned by the DHCP server, or (better), you could assign a dedicated IP address to your BMC within the DHCP server.

Then I suggest you configure the BMC with a fixed IP address, not an IP attributed by a DHCP server. If your LAN goes berserk and you have to connect to your BMC, it will be easier. Two choices : through the WebUI (example below) or through the UEFI BIOS.

Webui BMC network configuration

With the BMC configured, you have full control of your system through your favorite Web browser. HTML5 web Console Redirection is a fantastic feature not often available for free with other vendors (such as Dell, HPE or Lenovo for example).

PXE boot

It's of course possible to install your favorite OS by booting thanks to a USB stick. Having already an operational homelab, PXE boot is even easier and quicker. With the Supermicro a2sdi-4c-hln4f, there is just one caveat : no legacy boot available, only UEFI (well, the manual and options within the BIOS suggest legacy should be possible but in reality it's not working at all).

Through the UEFI BIOS options, you should verify the boot order or press F12 during the boot of the system and choose UEFI Network.