« Freenas with Xen » : différence entre les versions
mAucun résumé des modifications |
|||
(61 versions intermédiaires par le même utilisateur non affichées) | |||
Ligne 1 : | Ligne 1 : | ||
This howto is based on the excellent work made by [http://mywiredhouse.net/blog/building-freenas-xen-pvhvm-support/ My Wired House]. It aims at building [http://www.freenas.org/ FreeNAS] with PVHVM support and ultimately at putting it into production with PCI passthrough.<br /> | '''note''' : this tutorial applies to '''Freenas 9.3'''. Feenas 9.10 (based on FreeBSD 10) and upward natively includes the optimizations for Xen and LSI cards described here. | ||
This howto is based on the excellent work made by [http://mywiredhouse.net/blog/building-freenas-xen-pvhvm-support/ My Wired House]. It aims at building [http://www.freenas.org/ FreeNAS] with Xen PVHVM support and ultimately at putting it into production with PCI passthrough.<br /> | |||
== Build environment == | == Build environment == | ||
* Install | * Install Freebsd 9.3 (in a domU for example) | ||
* Make a few adjustements | * Make a few adjustements | ||
* Build Freenas | * Build Freenas | ||
=== Freebsd 9.3 in a domU === | === Installation of Freebsd 9.3 in a domU === | ||
* Install Freebsd 9.3 on a disk / volume of 100 Go (building Frenas requires space !) | |||
* Configuration file for Xl | |||
* | |||
<syntaxhighlight lang="python"> | <syntaxhighlight lang="python"> | ||
Ligne 90 : | Ligne 28 : | ||
# | # | ||
disk = [ | disk = [ | ||
'phy:/dev/system/freebsd | 'phy:/dev/system/freebsd-disk,xvda,w', | ||
'file:/opt/ISO/FreeBSD-9.3-RELEASE-amd64-bootonly.iso,hdc:cdrom,r' | |||
] | ] | ||
Ligne 117 : | Ligne 56 : | ||
stdvga=0 | stdvga=0 | ||
serial='pty' | serial='pty' | ||
usb= | usb=0 | ||
keymap='fr' | keymap='fr' | ||
localtime=1 | localtime=1 | ||
</syntaxhighlight> | </syntaxhighlight> | ||
* Note: To speed up the process, add RAM and vCPUs if you can ! | |||
=== Update & PVHVM configuration (to boost performances) === | |||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
freebsd-update fetch | |||
freebsd-update install | |||
reboot | |||
cd /usr/src | |||
make buildkernel KERNCONF=XENHVM | |||
make installkernel KERNCONF=XENHVM | |||
sed -i".backup" 's/ada/xbd/g' /etc/fstab | |||
sed -i".backup" 's/ifconfig_em0/ifconfig_xn0/' /etc/rc.conf | |||
reboot | |||
</syntaxhighlight> | </syntaxhighlight> | ||
==== Install Freenas dependencies | == Building Freenas == | ||
=== Install Freenas dependencies === | |||
Cf [https://github.com/freenas/freenas/tree/9.3-STABLE Readme on Github] | Cf [https://github.com/freenas/freenas/tree/9.3-STABLE Readme on Github] | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
pkg install poudriere-devel git cdrtools pxz python grub2-pcbsd xorriso py27-sphinx py27-sphinxcontrib-httpdomain-1.2.1 | |||
</syntaxhighlight> | </syntaxhighlight> | ||
=== | === Clone Freenas 9.3-STABLE branch === | ||
Clone the Freenas 9.3-STABLE branch from [https://github.com/freenas/freenas Github] | |||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
mkdir /home | |||
cd /home && git clone https://github.com/freenas/freenas.git | |||
cd freenas && git checkout 9.3-STABLE | |||
make checkout | |||
</syntaxhighlight> | </syntaxhighlight> | ||
=== A few adjustements on Freenas sources === | === A few adjustements on Freenas sources === | ||
* Modify "VERSION" in Makefile to reflect the custom release : | * Modify "VERSION" in ''Makefile'' to reflect the custom release : | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
sed -i".backup" 's/VERSION\?=9.3.1-STABLE/VERSION\?=9.3.1-STABLE-xen/ ; s/TRAIN\?=\${NANO_LABEL}-9.3-STABLE/TRAIN\?=${NANO_LABEL}-9.3-STABLE-xen/' /home/freenas/Makefile | |||
</syntaxhighlight> | </syntaxhighlight> | ||
* Edit /home/freenas | * Edit ''/home/freenas/build/nanobsd-cfg/FREENAS.amd64'' and at the end, add theses lines : | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
Ligne 177 : | Ligne 118 : | ||
</syntaxhighlight> | </syntaxhighlight> | ||
* Remove the "problematic" "set -e" in create_iso.sh (I know, this is bad but the | * Remove the "problematic" "set -e" in ''create_iso.sh'' (I know, this is bad but the process would stop at this stage if not ...) : | ||
<syntaxhighlight lang="bash"> | |||
sed -i".backup" 's/set -e/#set -e/' /home/freenas/build/create_iso.sh | |||
</syntaxhighlight> | |||
=== Build ! === | |||
Depending on the ressources allocated to your Freebsd build machine, it will take a while ! | |||
<syntaxhighlight lang="bash"> | |||
make release | |||
</syntaxhighlight> | |||
If all went well, you will find the result in ''/home/freenas/objs/FreeNAS-9.3-STABLE-xen-201506052152/x64'' : | |||
<syntaxhighlight lang="bash"> | |||
ls /home/freenas/objs/FreeNAS-9.3.1-STABLE-xen-201509111019/x64 | |||
FreeNAS-9.3-STABLE-xen-201509111019.GUI_Upgrade.txz FreeNAS-9.3-STABLE-xen-201509111019.iso | |||
FreeNAS-9.3-STABLE-xen-201509111019.GUI_Upgrade.txz.sha256.txt FreeNAS-9.3-STABLE-xen-201509111019.iso.sha256.txt | |||
</syntaxhighlight> | |||
== Installation of Freenas 9.3 in a domU == | |||
Installation is done through the following steps : | |||
* boot of the installer without PCI passthrough | |||
* first boot of the fresh Freenas installation without PCI passthrough | |||
* in Freenas, adding the necessary options for PCI passthrgouh ... And the last reboot with PCI passthrough activated | |||
=== Xl config file === | |||
<syntaxhighlight lang="python"> | |||
# | |||
# Kernel + memory size | |||
# | |||
builder = 'hvm' | |||
boot = 'cd' | |||
memory = '8192' | |||
vcpus = '2' | |||
# | |||
# Disk device(s). | |||
# | |||
disk = [ | |||
'phy:/dev/system/freenas-disk,xvda,w', | |||
'file:/opt/ISO/FreeNAS-9.3-STABLE-xen-201506052152.iso,hdb:cdrom,r' | |||
] | |||
# | |||
# Hostname | |||
# | |||
name = 'freenas' | |||
# | |||
# Networking | |||
# | |||
vif = [ 'mac=00:16:3e:xx:xx:xx, model=e1000' ] | |||
# | |||
# Behaviour | |||
# | |||
on_poweroff = 'destroy' | |||
on_reboot = 'restart' | |||
on_crash = 'restart' | |||
sdl=0 | |||
vnc = 1 | |||
vncunused = 1 | |||
vnclisten = '0.0.0.0' | |||
stdvga=0 | |||
serial='pty' | |||
usb=0 | |||
keymap='fr' | |||
localtime=1 | |||
</syntaxhighlight> | |||
=== Booting the installer === | |||
* Start the vm | |||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
xl create /etc/xen/freenas.ld | |||
</syntaxhighlight> | </syntaxhighlight> | ||
<!-- * As soon as the VM boots, open the console (e.g. via VNC), go to the Grub command prompt by pressing "c" and enter the following lines : --> | |||
<!-- <gallery> --> | |||
<!-- Fichier:Grub.png|Grub options to start the installer --> | |||
<!-- </gallery> --> | |||
* Follow the steps to install Freenas ; choose the volume named xbd0 which reflects the use of PV drivers | |||
=== Booting your fresh Freenas installation === | |||
<!-- Once again, because of this [https://bugs.freenas.org/issues/7376 little issue], booting the fresh Freenas installation for the first time requires special option ''hint.hpet.0.clock=0'' --> | |||
<!-- * As soon as the VM reboots after Freenas installation, reopen the console (e.g. via VNC), go to the Grub command prompt by pressing "c" and enter the following lines : | |||
<!-- <syntaxhighlight lang="bash"> --> | |||
<!-- grub> insmod zfs --> | |||
<!-- grub> kfreebsd /ROOT/default/@/boot/kernel/kernel --> | |||
<!-- grub> kfreebsd_loadenv /ROOT/default@/boot/device.hints --> | |||
<!-- grub> set kFreeBSD.kernelname="/boot/kernel/kernel" --> | |||
<!-- grub> kfreebsd_module /ROOT/default/@/boot/zfs/rpool.cache type=/boot/zfs/zpool.cache --> | |||
<!-- grub> set kFreeBSD.vfs.root.mountfrom=zfs:freenas-boot/ROOT/default --> | |||
<!-- grub> kfreebsd_module_elf /ROOT/default/@/boot/kernel/ispfw.ko --> | |||
<!-- grub> set kFreeBSD.hint.hpet.0.clock=0 --> | |||
<!-- grub> boot --> | |||
<!-- </syntaxhighlight> --> | |||
<!-- * In Freenas GUI, go to "System" -> "Tunales" -> "Add Tunable" : --> | |||
<!-- ** Variable : hint.hpet.0.clock --> | |||
<!-- ** Value : 0 | |||
<!-- ** Type : loader --> | |||
For PCI passthrough with LSI devices such as [http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-9211-8i.aspx LSI 9211-8i] (LSI 9211 cards in IT mode are recommanded with Freenas), two more tunables are required ; in Freenas GUI, go to "System" -> "Tunables" -> "Add Tunable" : | |||
* Variable : hw.pci.enable_msi | |||
** Value : 1 | |||
** Type : sysctl | |||
* Variable : hw.pci.enable_msix | |||
** Value : 0 | |||
** Type : sysctl |
Dernière version du 29 mars 2021 à 10:51
note : this tutorial applies to Freenas 9.3. Feenas 9.10 (based on FreeBSD 10) and upward natively includes the optimizations for Xen and LSI cards described here.
This howto is based on the excellent work made by My Wired House. It aims at building FreeNAS with Xen PVHVM support and ultimately at putting it into production with PCI passthrough.
Build environment
- Install Freebsd 9.3 (in a domU for example)
- Make a few adjustements
- Build Freenas
Installation of Freebsd 9.3 in a domU
- Install Freebsd 9.3 on a disk / volume of 100 Go (building Frenas requires space !)
- Configuration file for Xl
#
# Kernel + memory size
#
builder = 'hvm'
boot = 'cd'
memory = '2048'
vcpus = '1'
#
# Disk device(s).
#
disk = [
'phy:/dev/system/freebsd-disk,xvda,w',
'file:/opt/ISO/FreeBSD-9.3-RELEASE-amd64-bootonly.iso,hdc:cdrom,r'
]
#
# Hostname
#
name = 'freebsd'
#
# Networking
#
vif = [ 'mac=00:16:3e:xx:xx:xx, model=e1000' ]
#
# Behaviour
#
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
sdl=0
vnc = 1
vncunused = 1
vnclisten = '0.0.0.0'
stdvga=0
serial='pty'
usb=0
keymap='fr'
localtime=1
- Note: To speed up the process, add RAM and vCPUs if you can !
Update & PVHVM configuration (to boost performances)
freebsd-update fetch
freebsd-update install
reboot
cd /usr/src
make buildkernel KERNCONF=XENHVM
make installkernel KERNCONF=XENHVM
sed -i".backup" 's/ada/xbd/g' /etc/fstab
sed -i".backup" 's/ifconfig_em0/ifconfig_xn0/' /etc/rc.conf
reboot
Building Freenas
Install Freenas dependencies
pkg install poudriere-devel git cdrtools pxz python grub2-pcbsd xorriso py27-sphinx py27-sphinxcontrib-httpdomain-1.2.1
Clone Freenas 9.3-STABLE branch
Clone the Freenas 9.3-STABLE branch from Github
mkdir /home
cd /home && git clone https://github.com/freenas/freenas.git
cd freenas && git checkout 9.3-STABLE
make checkout
A few adjustements on Freenas sources
- Modify "VERSION" in Makefile to reflect the custom release :
sed -i".backup" 's/VERSION\?=9.3.1-STABLE/VERSION\?=9.3.1-STABLE-xen/ ; s/TRAIN\?=\${NANO_LABEL}-9.3-STABLE/TRAIN\?=${NANO_LABEL}-9.3-STABLE-xen/' /home/freenas/Makefile
- Edit /home/freenas/build/nanobsd-cfg/FREENAS.amd64 and at the end, add theses lines :
options NO_ADAPTIVE_MUTEXES
options NO_ADAPTIVE_RWLOCKS
options NO_ADAPTIVE_SX
# Xen HVM support
options XENHVM
device xenpci
- Remove the "problematic" "set -e" in create_iso.sh (I know, this is bad but the process would stop at this stage if not ...) :
sed -i".backup" 's/set -e/#set -e/' /home/freenas/build/create_iso.sh
Build !
Depending on the ressources allocated to your Freebsd build machine, it will take a while !
make release
If all went well, you will find the result in /home/freenas/objs/FreeNAS-9.3-STABLE-xen-201506052152/x64 :
ls /home/freenas/objs/FreeNAS-9.3.1-STABLE-xen-201509111019/x64
FreeNAS-9.3-STABLE-xen-201509111019.GUI_Upgrade.txz FreeNAS-9.3-STABLE-xen-201509111019.iso
FreeNAS-9.3-STABLE-xen-201509111019.GUI_Upgrade.txz.sha256.txt FreeNAS-9.3-STABLE-xen-201509111019.iso.sha256.txt
Installation of Freenas 9.3 in a domU
Installation is done through the following steps :
- boot of the installer without PCI passthrough
- first boot of the fresh Freenas installation without PCI passthrough
- in Freenas, adding the necessary options for PCI passthrgouh ... And the last reboot with PCI passthrough activated
Xl config file
#
# Kernel + memory size
#
builder = 'hvm'
boot = 'cd'
memory = '8192'
vcpus = '2'
#
# Disk device(s).
#
disk = [
'phy:/dev/system/freenas-disk,xvda,w',
'file:/opt/ISO/FreeNAS-9.3-STABLE-xen-201506052152.iso,hdb:cdrom,r'
]
#
# Hostname
#
name = 'freenas'
#
# Networking
#
vif = [ 'mac=00:16:3e:xx:xx:xx, model=e1000' ]
#
# Behaviour
#
on_poweroff = 'destroy'
on_reboot = 'restart'
on_crash = 'restart'
sdl=0
vnc = 1
vncunused = 1
vnclisten = '0.0.0.0'
stdvga=0
serial='pty'
usb=0
keymap='fr'
localtime=1
Booting the installer
- Start the vm
xl create /etc/xen/freenas.ld
- Follow the steps to install Freenas ; choose the volume named xbd0 which reflects the use of PV drivers
Booting your fresh Freenas installation
For PCI passthrough with LSI devices such as LSI 9211-8i (LSI 9211 cards in IT mode are recommanded with Freenas), two more tunables are required ; in Freenas GUI, go to "System" -> "Tunables" -> "Add Tunable" :
- Variable : hw.pci.enable_msi
- Value : 1
- Type : sysctl
- Variable : hw.pci.enable_msix
- Value : 0
- Type : sysctl