Why a High Availability DNS?

Because without DNS, nothing works. In corporate networks, infrastructure servers, like DNS, are run as HA or at least redundant by number. Usually, this is a feature of the app. But, we will do it with the humble Pi-hole.

How will this all work?

• Running 3 Pi-hole servers on different nodes, one preferably on real rust like a Raspberry Pi. They will use Unbound.
• One will be the master (192.0.2.53) from where all data is synchronized to the other two. Enter on one, use on many, performed by nebula-sync.
• The other two are used as a HA pair using one external IP 192.02.54. The HA Master is 192.0.2.55 and .192.02.56 will act as the HA Backup. Powered by keepalived

The Docker Installation of HA Pi-hole DNS

See my old post for more details: High Availability Pi-hole 6

Pi-hole version 6 – Docker image

The docker image has undergone a complete rewrite from the ground up, and is now based on Alpine rather than Debian. The same migration scripts that run on bare metal will also run on Docker – your configurations will be migrated to the new format.

The exception to this is environment variables. You can start the container with the old variables in place but don’t expect them to work! It is recommended to read the docker section of our docs page before upgrading.

Fresh Installation – Pi-hole and Unbound

For this version we are using Docker and the following apps:

• Pi-hole will be our interface for DNS, DHCP and Blocking of ads and bad IPs.
• Unbound is a validating, recursive, caching DNS resolver. It is designed to be fast and lean and incorporates modern features based on open standards.
  For redundancy and HA, we set up a Pi-hole on each node in the cluster or on several servers. This can also be a good project for all those unused Raspberry Pi everyone has in their drawers.
• Nebula-sync, synchronize Pi-hole v6.x configuration to replicas. It's not a part of the official Pi-hole project, but uses the API provided by Pi-hole instances to perform the synchronization actions.
• Keepalived, for high-availability, is achieved by using the VRRP protocol. VRRP is a fundamental brick for router failover.
• Docker on Alpine VM

Create the nodes

Roles by IP

• 192.0.2.53: pi-1, the master DNS, will be synced to the 2 HA nodes
• 192.0.2.54: The virtual IP for the HA DNS nodes
• 192.0.2.55: pi-2 HA master node
• 192.0.2.56: pi-3 HA node

For this example, we will set up the Pi-holes in Docker. Create an Alpine VM, add an ID and a name, enter the password and an SSH public key, select the Alpine ISO, 256–512M RAM, 8 G Disk and 1 CPU. Always use SSH keys for security.

Install Docker and Dockge on Alpine

See my post Docker + Dockge on an Alpine VM

While you can run Docker on a CT (LXC), the Proxmox team recommends that you use a VM rather than LXC.
While you can use an CT, I think it’s best to take the advice of the Proxmox team seriously for all production installs. Lab installs, who cares.

If you aren't going to use the Pi-hole for DHCP

• Set Network to DHCP and we are ready with the setup
• Set Firewall on
• Copy the MAC and set up your current DHCP to supply the correct IP and the other stuff a good DHCP can do

If you are using the Pi-hole for DHCP

Start the first VM 192.0.2.53

Upgrade the CT

apt-get update && apt-get dist-upgrade -y

Install other required parts

apt-get install ca-certificates curl gnupg -y

Add a user and add to sudo ans docker groups

adduser <username>

usermod -ag sudo <username>

Install Pi-hole

First, download the file and rename it.

wget -O pi-install.sh https://install.pi-hole.net 

Carefully read the file, change anything you don't like, and make sure you understand and accept the implications of running this script.

Make it executable, chmod 700 pi-install.sh or chmod +x pi-install.sh

Run the script ./pi-install.sh

• you will be asked to select your upstream DNS select Custom.

• I also recommend selecting the default StevenBlack's Unified Hosts List to be installed.
• Query logging on/off is your choice, read the documentation

Set your own password, pihole setpassword, otherwise use the random one as generated like AbDOlq83

Reboot and all is done for now, reboot now

Check for success

Login at the IP displayed to you, 192.0.2.53/admin

Install Unbound

apt install unbound -y

Configure Unbound

Copy the configuration file from the Unbound documentation and paste it into the configuration file pi-hole.conf. In my cleanup version, IPv6 is set to no.
You can modify it, there’s a lot that can be done with Unbound. For an initial set up we are using a generic setup.

cd /etc/unbound/unbound.conf.d && nano pi-hole.conf

After it's saved, restart the Unbound service.

service unbound restart

Check for success

First, by systemctl status unbound

Then, you test that Unbound is running properly by running the three commands below (see the Unbound documentation).

Test 1: dig pi-hole.net @127.0.0.1 -p 5335 should return an IP address

Test 2: Run this twice: dig fail01.dnssec.works @127.0.0.1 -p 5335 you should return SERVFAIL at the second time

Test 3: dig dnssec.works @127.0.0.1 -p 5335 should return NOERROR.

Configure Pi-hole to use Unbound

If you didn't already do it during the installation, do it now:

• In the GUI, go to Settings → DNS, uncheck all Upstream DNS Servers and add a Custom DNS server. In the entry field enter 127.0.0.1#5335

Making the DNS Highly Available

Create the other nodes

You should now create the two other Pi-hole servers, before you continue.
You will need their IP addresses and passwords later. They should have Pi-hole and Unbound as on the main one.

Keep Nodes in sync – Nebula-sync

Check their GitHub for the latest releases and download it

wget https://github.com/lovelaze/nebula-sync/releases/download/v0.3.0/nebula-sync_0.3.0_linux_amd64.tar.gz

Unpack

tar -xvzf nebula-sync_0.3.0_linux_amd64.tar.gz

Create the .env file.

Use CRON=* * * * * for testing, and CRON=0 * * * * for production.
Updating every minute is a bit excessive, hourly may be a better choice.

PRIMARY='http://ph1.example.com|password'
REPLICAS='http://ph2.example.com|password,http://ph3.example.com|password'
FULL_SYNC=true
CRON=* * * * *

Start nebula-sync

nebula-sync run --env-file .env

Run as a daemon

First, copy the executable to /usr/bin by cp nebula-sync /usr/bin then, you need to create a service file for systemd to be manage the service.

Create the service file

nano /etc/systemd/system/nebula-sync.service

[Unit]
Description=Nebula Sync Daemon

[Service]
ExecStart=/usr/bin/nebula-sync run --env-file /home/<user>/.env
WorkingDirectory=/home/<user>
Restart=always

[Install]
WantedBy=multi-user.target

Reload the systemd daemon to recognize the new service

sudo systemctl daemon-reload

Start your service

sudo systemctl start nebula-sync

Check for success

systemctl status nebula.service

Enable it to start on boot

sudo systemctl enable nebula-sync

High Availability with keepalived

Keepalived is designed to run on two separate hosts but share a virtual IP address. This ensures that if one goes down (the master), the backup will take over using the same virtual IP. In this example, the virtual IP is used as our backup DNS server.

Setup keepalived

Install Keepalived on both backup instances of Pi-hole (pi-2 and pi-3). Here we would like to have High Availability.

apt install keepalived

Get the interface name

ip a

Configure keepalived

Create and edit the file /etc/keepalived/keepalived.conf

Paste this information into the configuration file of the master and modify it as needed.

Create a password

Use any way you like, below 2 examples

< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c${1:-32};echo;

openssl rand -base64 32

Copy and edit the keepalived.conf files on pi-2 and pi-3

cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.orig && nano /etc/keepalived/keepalived.conf

The Keepalived Master – pi-2

Paste this into the keepalived.conf file, modify it with your password and IPs.

vrrp_instance VI_1 {
  state MASTER
  interface eth0
  virtual_router_id 54
  advert_int 1
  unicast_src_ip 10.10.1.55 
  unicast_peer {
    10.10.1.56
  }
  priority 20
  authentication {
    auth_type PASS
    auth_pass C35RUyBq
  }
  virtual_ipaddress {
    10.10.1.54/24    
  }
}

The Keepalived Backup – pi-3

Paste this into the keepalived.conf file, modify it with your password and IPs.

vrrp_instance VI_1 {
  state BACKUP
  interface eth0
  virtual_router_id 54
  advert_int 1
  unicast_src_ip 10.10.1.56
  unicast_peer {
    10.10.1.55  }
  priority 10
  authentication {
    auth_type PASS
    auth_pass C35RUyBq
  }
  virtual_ipaddress {
    10.10.1.54/24
  }
}

Enable the Keepalived service

Do this on both pi-2 and pi-3 on both instances

systemctl enable keepalived.service

Reboot

reboot now

Check for success

systemctl status keepalived.service

Test 1: Run a continuous ping to IP 192.0.2.54 and it should not break if you stop pi-2 or pi-3

Test 2: shutdown pi-2 and on pi-3 issue systemctl status keepalived.service and you should see a Entering MASTER STATE message

If you restart pi-2 you will see a Entering BACKUP STATE message.

DNS for LAN and VLAN

Basically, Pi-hole is a GUI for dnsmasq DNS and DHCP features.

If I add the following to /etc/dnsmasq.d/09-custom.conf DNS and DHCP should work on the eth0 interface and DHCP does start on the eth0.100 interface:

#VLAN 100 dirty DHCP and DNS congig
#router
dhcp-option=eth0.100,3,192.168.100.1

#dns server
dhcp-option=eth0.66,6,192.168.100.2

#dhcp range
dhcp-range=eth0.100,192.168.100.100,192.169.100.199,255.255.255.0,24h

References

Pi-hole ^[1] Unbound ^[2] Nebula-sync ^[3] Keepalived ^[4] CRON ^[5]
My old post ^[6]