« Hyperconverged Supermicro a2sdi-4c-hln4f » : différence entre les versions

De TeriaHowto
Sauter à la navigation Sauter à la recherche
mAucun résumé des modifications
mAucun résumé des modifications
Ligne 26 : Ligne 26 :
What is required for this project :
What is required for this project :
* mini tower format chassis
* mini tower format chassis
* Hypervisor OS on RAID 1 with two drives
* Hypervisor OS on mirror (RAID 1 in zfs terminology) with two drives
* at least 4 TB of encrypted data with a minimum of resilience (mirror or raidz1 / raidz2 ...)
* at least 4 TB of encrypted data with a minimum of resilience (mirror or raidz1 / raidz2 ...)
* it is recommended not to exceed 80% of the capacity of a ZFS filesystem, so, for 4 TB of usable data, 5 TB of raw capacity is required
* it is recommended not to exceed 80% of the capacity of a ZFS filesystem, so, for 4 TB of usable data, 5 TB of raw capacity is required
Ligne 37 : Ligne 37 :
* 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 4 * 2 TB SFF SSD (raidz1 storage)
* 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 4 * 2 TB SFF SSD (raidz1 storage)


The most reasonable would be to use LFF HDD but silent operation with a tiny chassis are very important, so let's stick with the "unreasonable" configuration !
The most reasonable would be to use LFF HDDs but silent operation in a tiny chassis with decent iops are very important, so let's stick with the "unreasonable" configuration !


=== Cooling the CPU ===
=== Cooling the CPU ===
Ligne 51 : Ligne 51 :
* [https://www.icydock.fr/goods.php?id=143 ToughArmor MB994SP-4SB-1] for data storage (with fans off) with 4 * [https://www.crucial.fr/ssd/mx500/ct2000mx500ssd1 Crucial MX500]
* [https://www.icydock.fr/goods.php?id=143 ToughArmor MB994SP-4SB-1] for data storage (with fans off) with 4 * [https://www.crucial.fr/ssd/mx500/ct2000mx500ssd1 Crucial MX500]
* [https://www.startech.com/fr-fr/cartes-additionelles-et-peripheriques/pexusb3s25 PCIe card with 2 * USB 3.0 ports]
* [https://www.startech.com/fr-fr/cartes-additionelles-et-peripheriques/pexusb3s25 PCIe card with 2 * USB 3.0 ports]
[[Fichier:Chassis_inside.jpg|none|thumb|600px|Inside the chassis]]
[[Fichier:Details_in_chassis.jpg|none|thumb|600px|Inside the chassis]]
== Installation ==
=== IPMI setup ===
First things first, taking control of BMC. These days, Supermicro's BMC are configured by default for DHCP and dedicated ethernet port. Just plug the dedicated port on a switch of your your LAN

Version du 31 décembre 2021 à 13:52

This howto aims at describing the choices and the build of a compact homelab with a hyperconverged chassis based on a Supermicro A2SDi-4C-HLN4F motherboard. The hypervisor OS will be Slackware64-current (with Qemu/KVM for virtualization), the storage will be provided by a Truenas core VM (thanks to pci-passthrough) and network orchestrated by an OPNSense VM.


Motivations

Why such a motherboard with a modest 4 cores Intel Atom C3558 ? Let's see the advantages :

  • obviously, it's very compact (mini-itx form factor)
  • up to 256 GB ECC RDIMM RAM supported
  • the CPU has a very low TDP (~ 17W), so no need for a fancy and potentially noisy cooling solution
  • 4 * 1 Gb/s Ethernet ports (cool for a network appliance such as OPNSense and there's no need for 10 Gb/s for this project)
  • dedicated IPMI Ethernet port
  • the SATA ports are provided by two distinct PCIe lines (see below, very important for pci-passthrough and no need for an additionnal HBA card)
A2SDi system block diagram

Well, the system has drawbacks too :

  • the CPU power will not be extraordinary
  • it's not possible to put a fan directly on top of the CPU heatsink (more on that later)
  • no external USB 3.0 ports (ony one "internal", on the motherboard itself) ; this is desirable to plug an external HDD for local backups (USB 2.0 is too slow)

Hardware

2.5 or 3.5 inches drives ?

What is required for this project :

  • mini tower format chassis
  • Hypervisor OS on mirror (RAID 1 in zfs terminology) with two drives
  • at least 4 TB of encrypted data with a minimum of resilience (mirror or raidz1 / raidz2 ...)
  • it is recommended not to exceed 80% of the capacity of a ZFS filesystem, so, for 4 TB of usable data, 5 TB of raw capacity is required
  • all disks easily accessible from the front of the chassis

Choosing between 2.5 (SFF) or 3.5 (LFF) inches drives is not so easy. Obviously, for raw capacity over a gigabit network, LFF HDD are unbeatable in terms of price. In France (as of march 2021), it costs ~ 80 euros for a 2 TB NAS 3.5 LFF HDD ... Same price for a 1 TB NAS SFF HDD. Moreover, SFF HDD over 2 TB simply doesn't exist for consumer NAS systems (enterprise SFF HDDs exist but are way too expensive). LFF HDD are a clear winner ? What about chassis size, cooling and noise ?

At this point, there are two choices :

  • 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 2 * 6 TB LFF HDD (mirror storage)
  • 2 * 500 GB SFF SSD (RAID 1 for hypervisor and VMs OS) plus 4 * 2 TB SFF SSD (raidz1 storage)

The most reasonable would be to use LFF HDDs but silent operation in a tiny chassis with decent iops are very important, so let's stick with the "unreasonable" configuration !

Cooling the CPU

As seen earlier, a standard CPU fan cannot be fitted on the heat sink.

final system

Inside the chassis
Inside the chassis

Installation

IPMI setup

First things first, taking control of BMC. These days, Supermicro's BMC are configured by default for DHCP and dedicated ethernet port. Just plug the dedicated port on a switch of your your LAN