Hyperconverged Supermicro a2sdi-4c-hln4f

This howto aims at describing the choices and the build of a compact homelab with a hyperconverged chassis based on a Supermicro A2SDi-4C-HLN4F motherboard. The hypervisor OS will be Slackware64-current (with Qemu/KVM for virtualization), the storage will be provided by a Truenas core VM (thanks to pci-passthrough) and network orchestrated by an OPNSense VM.

Motivations

Why such a motherboard with a modest 4 cores Intel Atom C3558 ? Let's see the advantages :

• obviously, it's very compact (mini-itx form factor)
• up to 256 GB ECC RDIMM RAM supported
• the CPU has a very low TDP (~ 17W), so no need for a fancy and potentially noisy cooling solution
• 4 * 1 Gb/s Ethernet ports (cool for a network appliance such as OPNSense and there's no need for 10 Gb/s for this project)
• dedicated IPMI Ethernet port
• the SATA ports are provided by two distinct PCIe lines (see below, very important for pci-passthrough and no need for an additionnal HBA card)

A2SDi system block diagram

Well, the system has drawbacks too :

• the CPU power will not be extraordinary
• it's not possible to put a fan directly on top of the CPU heatsink (more on that later)
• no external USB 3.0 ports (ony one "internal", on the motherboard itself) ; this is desirable to plug an external HDD for local backups (USB 2.0 is too slow)

Hardware

2.5 or 3.5 inches drives ?

What is required for this project :

• mini tower format chassis
• Hypervisor OS on mirror (RAID 1 in zfs terminology) with two drives
• at least 4 TB of encrypted data with a minimum of resilience (mirror or raidz1 / raidz2 ...)
• it is recommended not to exceed 80% of the capacity of a ZFS filesystem, so, for 4 TB of usable data, 5 TB of raw capacity is required
• all disks easily accessible from the front of the chassis

Choosing between 2.5 (SFF) or 3.5 (LFF) inches drives is not so easy. Obviously, for raw capacity over a gigabit network, LFF HDD are unbeatable in terms of price. In France (as of march 2021), it costs ~ 80 euros for a 2 TB NAS 3.5 LFF HDD ... Same price for a 1 TB NAS SFF HDD. Moreover, SFF HDD over 2 TB simply doesn't exist for consumer NAS systems (enterprise SFF HDDs exist but are way too expensive). LFF HDD are a clear winner ? What about chassis size, cooling and noise ?

At this point, there are two choices :

• 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 2 * 6 TB LFF HDD (mirror storage)
• 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 4 * 2 TB SFF SSD (raidz1 storage)

The most reasonable would be to use LFF HDDs but silent operation in a tiny chassis with decent iops are very important, so let's stick with the "unreasonable" configuration !

Cooling the CPU

As seen earlier, a standard CPU fan cannot be fitted on the heat sink.

final system

• Chieftec mini tower case
• Supermicro A2SDi-4C-HLN4F
• 64 GB DDR4 RAM ECC (RDIMM)
• ToughArmor MB992SK-B for hypervisor and VMs OS with 2 * WD SA500
• ToughArmor MB994SP-4SB-1 for data storage (with fans off) with 4 * Crucial MX500
• PCIe card with 2 * USB 3.0 ports

Inside the chassis

Inside the chassis

Installation

IPMI setup

First things first, taking control of BMC. These days, Supermicro's BMC are configured by default for DHCP and dedicated ethernet port. Just plug the dedicated port on a switch of your your LAN