HBA or IT-mode

For running ZFS on RAID controllers we need the RAID-controller in HBA or IT-mode for RAID pass trough and a separate boot device, you can't usually boot from a HBA device.

HP/HPE Proliant servers usually contain a hardware-based RAID controller (e.g. P410i or P420i controller). This article explains one way of how to disable the RAID modes of this controller and use the attached disks directly.

First list your device with details lspci -Dmmnn | grep LSI or RAID

You may want to avoid hardware RAID for many reasons:

• you need the superior features and the speed of ZFS
• you want software RAID instead, no lock-in with regard to a RAID controller
• you require TRIM/DISCARD for SSD's, but the controller lacks this feature
• you need direct access to the disk internals via smartctl
• you are running out of space
• you want self healing and no more 'bit rot'
• you do not want or you are not allowed to install the proprietary management software (provided as RPM packages) from HP, but still require access to the disk internals
• your backup strategy is built around ZFS and its features
• you do not have a valid iLo licence

Disabling the RAID controller is not trivial

• You cannot simply remove the controller, since it provides the hardware connector for attaching the front disk bays (besides the RAID controller there is only one on-board SATA connector usually for the DVD drive).
• Configuring RAID 0 for all disks separately doesn't work (it looks it's working but it's not). Reason: these disks still contain the meta data format of the RAID controller (thus they are not usable without this controller) and there is still no access via smartctl.

RAID 0 is not a fix - but sometimes the only but dangerus way

Downsides with the controller in HBA mode (pass-through)

• BIOS cannot boot from the disks attached to the RAID controller
• you cannot obviously use the RAID modes provided by the controller
• if the disks currently contain data, you should backup before the move
• it obviously affects all the disks on the controller

Add the HP SSA CLI tools to Proxmox node's

Proxmox is running Debian. For HPE Gen8 or later servers, we need to prepare the system. First list your devices with lspci or lspci -knn. Then run the 3 steps below to install HP Smart Storage Administrator (HP SSA) CLI for Linux 64-bit.

You could try the Offline HP Smart Storage Administrator Utility iso copied to a USB memory stick and boot in UEFI. [Download]

Older HP and HPE controllers was newer built for HBA or IT-mode.

Step 1 - Add a new repository to the sources.list.

nano /etc/apt/sources.list ## add the row deb http://downloads.linux.hpe.com/SDR/repo/mcp bullseye/current non-free

You will see the repository in the GUI and you can disable it

Step 2 - Add the 3 GPG keys

curl http://downloads.linux.hpe.com/SDR/hpPublicKey2048.pub | apt-key add -; curl http://downloads.linux.hpe.com/SDR/hpPublicKey2048_key1.pub | apt-key add -; curl http://downloads.linux.hpe.com/SDR/hpePublicKey2048_key1.pub | apt-key add -

Step 3 - Update and install ssacli tools

apt update && apt install ssacli -y ## Find the slot and type of RAID card ssacli ctrl all show status ## For controllers starting with P: #ssacli controller slot=0 modify hbamode=on ##For controllers starting with H: #ssacli controller slot=0 modify raidmode=off ## Verify the result (if required) #ssacli controller slot=0 show

Booting without the RAID controller

The Proliant BIOS and/or the RAID controller do have the following limitations with regard to the boot medium

Limitations of HBA mode

• RAID controllers don't support booting from the disks in HBA mode
• Booting from the mainboard's internal SD card the BIOS provides access only to that SD card. It's not sufficient to simply dump a minimal grub bootloader onto the SD card - it will not find the root filesystem containing the other grub modules and configuration

Workaround

• place the /boot directory on a filesystem on the SD card

Use a high quality memory card or USB-stick
and make a copy to have for emergency situations

• optionally with a software RAID ( metadata version 0 - for transparency ) connected to the real system hard disks attached to the RAID controller1.

Installing LSI MegaCli utility on Proxmox

Setting up and manage LSI HBA and RAID controllers and their disks.
Broadcoms pages have more info and a release document

1. Install necessary tools apt update && apt-get install unzip alien
2. Install necessary lib apt install libncurses5
3. Download wget https://docs.broadcom.com/docs-and-downloads/raid-controllers/raid-controllers-common-files/8-07-14_MegaCLI.zip
4. Unzip unzip 8-07-14_MegaCLI.zip
5. Create debian package cd Linux and alien MegaCli-8.07.14-1.noarch.rpm
6. Install debian package dpkg -i megacli_8.07.14-2_all.deb
7. Run MegaCli with -h for a menu /opt/MegaRAID/MegaCli/MegaCli64 -h