Arch Linux server quick install guide

I install Arch Linux maybe once every 3 years. It can be annoying to have to navigate through all the relevant ArchWiki pages to find the pieces of information I need. So this post lists the instructions I followed when I last installed Arch on a server with:

• Unified Kernel Image
• Secure Boot
• Encrypted rootfs
• TPM2 auto-unlocking
• Firewall

I generally try not to deviate too much from the default settings, so aside from the nftables config, you can expect most things to be configurable by just uncommenting a few lines here and there.

1 Preparation

1. Download the Arch Linux ISO and write it to a USB drive:

   $ cat arch.iso > /dev/sdx
   

2. Put Secure Boot in Setup Mode so you can enroll your own keys.

3. Boot from the USB drive.

2 Live environment

Formatting the disk and installing the base system.

2.1 Partition the disk

Start fdisk to partition the disk (replace /dev/sdz with the actual block device path):

$ fdisk /dev/sdz

Then create a GPT partition table with the following partitions:

• EFI system partition (512 MiB) (type 1).
• Linux root (x86-64) (type 23).

2.2 Format the partitions

Format the EFI system partition with FAT32:

$ mkfs.fat -F 32 /dev/sdz1

Encrypt the root partition with LUKS:

$ cryptsetup luksFormat /dev/sdz2

Decrypt the root partition and enable TRIM permanently:

$ cryptsetup --allow-discards --persistent open /dev/sdz2 root

Format the root volume with Btrfs:

$ mkfs.btrfs /dev/mapper/root

2.3 Mount the partitions

Mount the root partition to /mnt:

$ mount /dev/mapper/root /mnt

Mount the EFI system partition to /mnt/efi:

$ mount --mkdir /dev/sdz1 /mnt/efi

2.4 Install the base system

Connect to the network (wired + DHCP works out of the box), then bootstrap the system:

$ pacstrap -K /mnt base linux linux-firmware util-linux intel-ucode \
    dosfstools btrfs-progs sbctl efibootmgr neovim

Add iwd if a wireless connection is needed.

2.5 Generate the fstab

Auto-generate the fstab:

$ genfstab -U /mnt >> /mnt/etc/fstab

Edit /mnt/etc/fstab to replace the mount options with defaults,noatime. For /efi, you can add fmask=0077,dmask=0077 so systemd-boot doesn’t complain that /efi/loader/random-seed is world-readable.

3 Chroot

Change root to the new system:

$ arch-chroot /mnt

3.1 Base config

Configuring localization and hostname.

3.1.1 Timezone

Set the timezone:

$ ln -sf /usr/share/zoneinfo/Europe/Paris /etc/localtime

Synchronize the hardware clock:

$ hwclock --systohc

3.1.2 Locale

Uncomment en_US.UTF-8 in /etc/locale.gen and generate the locales:

$ locale-gen
Generating locales...
  en_US.UTF-8... done
Generation complete.

3.1.3 Keymap

Set the console keymap:

$ echo KEYMAP=us >> /etc/vconsole.conf

3.1.4 Hostname

Set the hostname:

$ echo whatever >> /etc/hostname

3.2 Configure the boot image

Kernel, cmdline, initramfs, microcode, splash image bundled into a single Unified Kernel Image that can be natively booted by a UEFI firmware, no additional bootloader needed.

3.2.1 Encrypted rootfs hooks

Edit /etc/mkinitcpio.conf hooks to support microcode updates and encrypted rootfs:

HOOKS=(base systemd autodetect microcode modconf kms keyboard sd-vconsole sd-encrypt block filesystems fsck)

3.2.2 Unified Kernel Image (UKI)

Edit /etc/mkinitcpio.d/linux.preset to build a UKI:

ALL_kver="/boot/vmlinuz-linux"

PRESETS=('default' 'fallback')

default_uki="/efi/EFI/Linux/arch-linux.efi"
default_options="--splash /usr/share/systemd/bootctl/splash-arch.bmp"

fallback_uki="/efi/EFI/Linux/arch-linux-fallback.efi"
fallback_options="-S autodetect"

3.3 Setup Secure Boot

Create the Secure Boot keys:

$ sbctl create-keys
Created Owner UUID bfacc526-9b2b-4752-a8b2-add202b7b6c7
Creating secure boot keys...✓
Secure boot keys created!

Enroll them:

$ sbctl enroll-keys -m
Enrolling keys to EFI variables...
With vendor keys from microsoft...✓
Enrolled keys to the EFI variables!

3.4 Build the boot images

Create the EFI/Linux directory:

$ mkdir -p /efi/EFI/Linux

Build the boot images:

$ mkinitcpio -P

Verify that the images are properly signed (done automatically by the /usr/lib/initcpio/post/sbctl hook):

$ sbctl verify
Verifying file database and EFI images in /efi...
✓ /efi/EFI/Linux/arch-linux-fallback.efi is signed
✓ /efi/EFI/Linux/arch-linux.efi is signed

3.5 Create the boot entries

Create the main boot entry:

$ efibootmgr --create --disk /dev/sdz --part 1 --label "Arch Linux" \
    --loader '\EFI\Linux\arch-linux.efi' --unicode

Create the fallback boot entry:

$ efibootmgr --create --disk /dev/sdz --part 1 --label "Arch Linux (fallback)" \
    --loader '\EFI\Linux\arch-linux-fallback.efi' --unicode

You can also delete existing entries:

$ efibootmgr --bootnum 0001 --delete-bootnum

And modify the boot order:

$ efibootmgr --bootorder 0000,0001

See efibootmgr.

3.6 Reboot

Set the root password:

$ passwd

Reboot the system:

$ reboot

4 First boot

If the system booted successfully, at least Secure Boot is working.

4.1 TPM2 unlocking

Install tpm2-tss:

$ pacman -S tpm2-tss

Enroll a new key with TPM2 unlocking (replacing any existing TPM2 slots):

$ systemd-cryptenroll --tpm2-device=auto --wipe-slot=tpm2 /dev/sdz2

4.2 Setup networking

Configure DHCP for the wired interfaces:

[Match]
Name=en*

[Network]
DHCP=ipv4

[DHCP]
RouteMetric=10
UseDNS=no

Link /etc/resolv.conf to the systemd-resolved stub:

$ ln -sf /usr/lib/systemd/resolv.conf /etc/resolv.conf

Enable the network services:

$ systemctl enable --now systemd-networkd systemd-resolved systemd-timesyncd

4.3 Configure the system services

Install irqbalance if you believe in evenly distributed IRQs:

$ pacman -S irqbalance

Set a journal size limit:

[Journal]
SystemMaxUse=100M

Configure system watchdogs:

[Manager]
RuntimeWatchdogSec=15s
RebootWatchdogSec=3min

Enable system services:

$ systemctl enable --now fstrim.timer irqbalance systemd-oomd

By default Btrfs also issues asynchronous discards (if discard=async doesn’t appear in the mount options, then TRIM is misconfigured):

$ findmnt /
TARGET SOURCE           FSTYPE OPTIONS
/      /dev/mapper/root btrfs  rw,noatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/

4.4 Setup OpenSSH

Install the package:

$ pacman -S openssh

Configure public key authentication (permissions are important):

$ mkdir -m 700 ~/.ssh
$ echo 'ssh-ed25519 AAAA...' >> ~/.ssh/authorized_keys
$ chmod 600 ~/.ssh/authorized_keys

Enable the server:

$ systemctl enable --now sshd

4.5 Setup a firewall

Install iptables-nft:

$ pacman -S iptables-nft

Allow core network services and SSH:

#!/usr/bin/nft -f

destroy table inet firewall
table inet firewall {
    chain input {
        type filter hook input priority filter; policy drop;

        ct state established,related accept
        ct state invalid drop

        iifname lo accept

        icmp type echo-request accept
        icmpv6 type { echo-request, nd-router-advert, nd-router-solicit, nd-neighbor-advert, nd-neighbor-solicit } accept

        tcp dport llmnr accept
        udp dport llmnr accept

        tcp dport ssh accept
    }

    chain forward {
        type filter hook forward priority filter; policy drop;
    }
}

Enable the firewall:

$ systemctl enable --now nftables

4.6 Performance Tuning

The default LUKS settings related to I/O queues are suboptimal for NVMe SSDs. The read/write workqueues can be permanently disabled using the following command (with --persistent you must pass all the flags you want enabled, which is why I also added --allow-discards):

$ cryptsetup \
  --allow-discards \
  --perf-no_read_workqueue --perf-no_write_workqueue \
  --persistent refresh root

Dumping the LUKS header should show these flags are enabled:

$ cryptsetup luksDump /dev/sdz2 | grep Flags
Flags:          allow-discards no-read-workqueue no-write-workqueue

5 Next steps

1. Reboot to check that everything works.
2. Install Tailscale, k3s, whatever.
3. Setup smartmontools, btrfs-scrub@-.timer, backups.