HP Smart Array 410i as HBA

ZFS needs an HBA, but HP is shy on the older devices. As I'm having an older RAID card, to get access to all settings on the raid card. Will achieve by using a third party hack.

This is about how I did it. I have 4 x 900 G SAS drives as a fast ZFS2 array and a PCIe SSD for boot and systems, it also has a 4th partition used as a single disk ZFS for some HA VM's with replication.

This is not a guide, it's a concept description

This is dangerous stuff to do - think about this warning for a while:

It's possible to use RAID controllers to set up 4 raid0 disks, but some cards need HP's tools/licenses to do so. It will work, but there are some issues, and that's why everyone is saying don't do it.
You also need to know how to set up your BIOS and RAID bios.

Some issues to fix before getting started

Check your hardware and firmware before starting. See documentation or run Some commands like hwinfo --storage or use lspci, hardinfo, inxi etc.

Ask your self:
- Do I have the skills needed if it goes sideways
- Do I want to risk a 100 - 600€ server doing this
- Should I get an IT-mode/HBA card for 30 - 70€

My issues were

1. I needed a fast SAS storage
2. I needed the PCIe ports and couldn't add an HBA
3. Old raid firmware version needed updating
4. No HP tools for Proxmox or Debian

Start the upgrade process for P410i

Upgrade to 6.64.2, or later. Download the firmware upgrade file, from HPE's Red Hat based Linux repository. Smart Component provides firmware for the following storage controllers: Smart Array P212, P410, P410i, P411, P711m, P712m, and P812.

wget https://downloads.hpe.com/pub/softlib2/software1/sc-linux-fw-array/p332076214/v110820/hp-firmware-smartarray-14ef73e580-6.64-2.i386.rpm

To be able to use the .rpm on Debian, we can use a tool called rpm2cpio

apt update && apt install rpm2cpio libc6-i386 lib32stdc++6

We also need, use apt update before installing

apt install hwinfo git dpkg-dev 

# You might need
#apt install compat-libstdc++-33.i386 compat-libstdc++-33.x86_64 compat-libstdc++-296.i386 

# These are needed, but you might already have them 
#apt install curl coreutils gcc kmod make patch

#systemctl reboot

# Adjust according to your version: uname -r
apt install pve-headers

#  Get the kernel headers on Debian based systems
#apt dist-upgrade -y
#systemctl reboot
#apt install linux-headers-$(uname -r)
#apt upgrade linux-headers-$(uname -r)

Extract the binary

rpm2cpio  hp-firmware-smartarray-14ef73e580-6.64-2.i386.rpm  | cpio -idmv

Copy ccissflash binary to somewhere under your PATH (echo $PATH)

cd usr/lib/i386-linux-gnu/hp-firmware-smartarray-14ef73e580-6.64-2/ 
cp ccissflash  /usr/local/bin

Run the upgrade binary‌

bash ccissflash 
#./ccissflash 
# bash hpsetup 
# ./hpsetup

Depending on your setup, you might need to run bash hpsetup or ./hpsetup

Final step is to reboot and check everything for errors

Now we will set up the HP Smart Array HBA tool

If the firmware upgrade is successful, you should be able to reboot into your server in some minutes. And verify the version of your current running firmware.

If you have a P410(i) in HBA, you need to patch your kernel.

For Ubuntu, the 5.18 kernels are on this page. I tested with Ubuntu.

apt install git dpkg-dev
apt install pve-headers

Now patch the kernel (5.18.83-1 as an example)

git clone https://github.com/im-0/hpsahba.git 

cd hpsahba 

# Create the man pages
apt install pandoc -y 
make

Check if this folder is present on your system or adapt according to your kernel /lib/modules/5.15.85-1-pve/build

cd hpsahba/contrib/dkms 

cp Makefile Makefile.orig

nano Makefile
# Do this change, here is my version of kernel:

#KDIR := /usr/lib/modules/$(KRELEASE)/build 
KDIR := /lib/modules/5.15.85-1-pve/build

Add a symlink /var/lib/dkms/hpsa-dkms/1.0/source → /usr/src/hpsa-dkms-1.0

dkms add ./

Add the hpsa-dkms to your kernel

dkms install --force hpsa-dkms/1.0

Start using it

modprobe -r hpsa
modprobe hpsa hpsa_use_nvram_hba_flag=1
echo "options hpsa hpsa_use_nvram_hba_flag=1" > /etc/modprobe.d/hpsa.conf
update-initramfs -u

Output example

Restart ‌your node

init 6

After the re-boot, you should see your disks as separate disks

lsblk

cd hpsahba/
./hpsahba -i /dev/sg0

Start using SMART on your disks - if possible

SMART is basically for ATA ans SATA devices. SCSI, SAS and NVMe drives do not always provide ATA/SATA-like SMART Attributes.

Check your HW by batcat /proc/scsi/scsi

smartctl --scan-open             # Check for complient disks
smartctl -a /dev/sdg0            # For P410i you see no S.M.A.R.T.
smartctl -a -d cciss,0 /dev/sg0  # Gives some info from thr card
# If your disk is supported
smartctl -a /dev/sda             # Reads the info from the disk
smartctl -s on /dev/sda          # Start monitoring in Proxmox

Conclusion

To summarize a week's work

Was it hard?

Lots of work finding the patches, hours of fact finding and recovery actions.

Did the installation go smoothly?

Not at all. Endless reboots that take forever (servers are like that) and installing missing parts and doing compromises. And more fact finding and learning.

Did it go sideways?

Yes it did. Murphy's law, ones again.
I did installation bat to read the logs, it helps.

apt update && apt install bat -y

# Add the use of my aliases and prompt in bash
nano .bashrc
...
[[ -f ~/.bash_aliases ]] && . ~/.bash_aliases

# Add a alias to use bat as on all other systems
nano .bash_aliases
...
alias bat='batcat'

Was it wort the effort?

As a learning experience - absolutely YES
Counting the risk and time spent - definitely NO.
Will I do it again - NO, for the next server I will get an HBA

Is it working?

Yes, it's up and running with 4 SAS drives in a fast ZFS2 array.

References

How to setup servers is one of my post diskussing some aspects of things mentiond in this post ^[1]
To make a RAID controller to HBA we need tools. HP Smart Array HBA tool ^[2]
See Proxmox forums there is tons of info on anything and everything ^[3] and ^[4]
The Art of Server has a web shop for HBA's and an nice YouTube channel ^[5]
Smartmontools wiki has the answers to SMART questions ^[6]