Unraid (Holly)¶

Overview¶

Hostname: Holly
IP: 192.168.1.200 (SSH on port 22159)
URL: holly.mdhmedia.uk
Role: Primary NAS, media storage, Docker host
Platform: Unraid 7.2.5 — runs as VM 110 on pve2 (USB boot, PCI passthrough), not bare metal
Containers: managed via Unraid Docker UI / XML templates (Compose Manager plugin installed but unused)

Fully recovered — 2026-07-29

The 2026-07-25 boot-USB failure (fixed by reseating the flash drive on 2026-07-26) had also corrupted docker.img; the Docker daemon came back after a VM reboot on 2026-07-29 and Holly is fully operational again.

Backups¶

vzdump can't cover Holly (USB boot + passthrough disks — appdata lives on the physical array, invisible to the hypervisor), so since 2026-07-31 a static proxmox-backup-client runs inside Holly and ships appdata + the boot flash to the PBS datastore as host/holly. Hand-deployed to /boot/config/plugins/holly-pbs-backup/ (plus a dynamix cron file); the PBS token/fingerprint live in a Holly-local pbs.env, not in git. See holly/backup/ in the repo.

Time	Step
06:00	Consistent staging copy — containers quiesced, appdata rsynced to a cache-SSD staging area
20:35	Upload of staging + `/boot` to `host/holly` on PBS, late in the WoL wake window (moved from 19:25 on 2026-08-03 — pve3 is load-marginal under concurrent load; the job exits quietly if PBS is asleep)

The nightly PBS → Hetzner offsite sync carries host/holly along automatically, and the first successful appdata backups are in the datastore (the Jarvis 20:05 backup-watchdog tracks their age — ~23.5 h at check time is healthy). Unraid Connect's cloud flash backup remains as the second, independent copy of the boot flash.

Docker Containers¶

Inventory as of 2026-07-03 (all on the default bridge unless noted); versions are point-in-time and may lag the live Unraid Docker UI:

Container	Version	Port	URL	Purpose
NZBGet	v26.2-ls250	6789	nzbget.mdhmedia.uk	Usenet downloader
Prowlarr	2.4.0.5397	9696	prowlarr.mdhmedia.uk	Indexer manager
Radarr	6.2.1.10461	7878	radarr.mdhmedia.uk	Movie automation
Sonarr	4.0.19.2979	8989	sonarr.mdhmedia.uk	TV automation
qBittorrent	5.2.1	8080	torrent.mdhmedia.uk	Torrent client
Ombi	v4.53.10	3579	ombi.mdhmedia.uk	Media requests
Plex	1.43.2.10687	32400	— (route commented out)	Media server — dedicated br0 IP 192.168.1.210
Speedtest Tracker	v1.14.5	8084	speedtest.mdhmedia.uk	Network speed tracking
Audiobookshelf	2.35.1	13378	abs.mdhmedia.uk	Audiobook server
Libation	retired 2026-08-10	—	libate.mdhmedia.uk	GUI container retired; liberation now runs headless as libation-sync on CT 101
iperf3	—	host net	—	Throughput testing

Not running: webtop (created, never started), abs-organiser (exited Apr 2026). Stale XML templates with no container behind them: GrafanaLoki, NginxProxyManager, cloudflared, cloudflareddns, jackett, netvisor-daemon.

Storage¶

Minimum Free Space — the setting that actually breaks writes¶

On a JBOD array each file lives whole on one disk, so a disk being 95% full does not stop writes: shfs routes the next file elsewhere. What does break a write is Minimum Free Space being smaller than the file being written — shfs starts the write on a disk that cannot finish it.

Audited and corrected 2026-08-10:

Share	Was	Now	Why
`Media`	23.8 GiB	100 GiB	91 files exceed the old floor; the largest on the array is 83.2 GiB (`Interstellar` 00001.m2ts). An 83 GiB remux could be started on a disk with 25 GiB left
`onedrive`	0	20 GiB	A floor of 0 lets shfs fill a disk to zero mid-file. The share is empty today, so this was latent
`downloads`	372 GiB	unchanged	Looks absurd next to a 3.36 GiB largest file, but it is protective: it keeps a churning share off the nearly-full disks, confining it to disk3/disk5

Rebalance — 2026-08-11 (~460 GB moved, disk1 + disk4 → disk3)¶

Moved four whole TV-show folders (Frasier, House of the Dragon ×2, One Piece) off the two fullest disks onto disk3, which had by far the most headroom. Unbalance was installed for this, but its scatter/gather engine turned out to be an undocumented WebSocket protocol (scatter:plan:start / move / copy commands, no REST API) — not something to reverse-engineer live against production data. The move was done directly instead, with the same discipline: rsync -a --checksum to a temporary name on the target disk → independent file-count + byte-count verification → source removed only after verification passes → verified copy renamed into its final place. Array health re-checked before and after every item.

Disk	Before	After
disk1	219 GB free	371 GB free
disk3	1.3 TB free	852 GB free
disk4	222 GB free	503 GB free

The mv-into-existing-directory pitfall

mv SRC DST where DST already exists as a directory nests SRC inside it (DST/basename(SRC)) rather than merging — and does this silently, with no error, if the nested name doesn't collide with anything yet. That's what happened moving House of the Dragon off disk4: the show already had unrelated content on disk3 (a pre-existing Season 3 and one Season 2 episode from the normal split-level allocator), so $final already existed, and the disk4 portion landed nested one level down without complaint. It only became visible when the second House-of-the-Dragon move (from disk1) tried the same nested name and it was now occupied - mv: cannot overwrite '.../House of the Dragon/House of the Dragon': File exists.

Nothing was ever at risk (every deletion happens only after independent verification), but the final structure needed a manual, collision-checked merge: move the nested content's files into the flat path one season at a time, verifying no filename collision first.

A file's mtime does not prove when it arrived at a path - mv preserves the original mtime, so old dates on files sitting somewhere unexpected do not mean "pre-existing junk". Reconcile by name+size (or a checksum) against what the move actually reported, not by how old the files look.

Any future script driving this kind of move must merge into an existing target directory explicitly (mv SRC/* DST/, after a collision check) rather than mv SRC DST and trust it either renames cleanly or errors.

How it was applied

The supported route is the webGUI (POST /update.htm), but an unauthenticated POST 302s to /login. Instead both files emhttpd keeps in sync were edited directly, with timestamped backups: /boot/config/shares/<name>.cfg (shareFloor=, persistent, read at array start) and /var/local/emhttp/shares.ini (floor=, live — shfs inotify-watches it, so no array restart was needed). Safe because emhttpd generates shares.ini from the .cfg, so setting both to the same value survives any regeneration. Backups: *.bak-2026-08-10-225809.

Unraid has no python3 — use PHP for scripted edits on the box.

Health + capacity status (2026-07-30)

All disks are SMART-healthy — 0 reallocated sectors, 0 pending, 0 CRC errors — though two data drives are at ~54,500 power-on hours (~6.2 years).
On 2026-08-10 disk1 + disk2 dropped inside the VM for ~1 h 45 m after a vfio-pci controller reset. This was not drive failure: all six disks SMART-passed when they returned. Zero-parity Unraid reported DISK_OK, so nothing alerted; the UnraidArrayDiskMissing Prometheus alert now covers it.
The array is deliberately zero-parity: the "2 disabled" slots shown in the UI are simply the empty parity slots, not failed drives.
The real issue is capacity: disk4 96%, disk1 95%, disk6 93%, disk2 90%.
263.9 GB of byte-verified duplicates were deleted on 2026-07-28 — duplicate movie downloads 146.5 G + audiobook duplicates 95.6 G + redundant caab backups 21.8 G (the GoPro and OneDrive trees were kept).

Array 91% full (analysed 2026-07-06)

/mnt/user: 24T used. The fill is almost entirely legitimate Media (22T) — TV Shows 12T, Movies 9.6T — so there is no junk to delete; the real fix is the NAS rebuild. The originally-suspected junk shares (DVDRips, tdarrcache, lancache, fbshare) are already empty (0 bytes).

Short-term reclaim candidates (share-by-share du):

Target	Size	Note
`downloads/` completed rips	~300–400 G	2160p imports already in Media (real copies, `nlink=1`) — but many are actively seeding in qBittorrent; prune per-file, not wholesale
`system/docker` on the array	316 G	two images (`docker.img` 100 G + `docker1.img` 250 G — likely one orphaned); ideally lives on cache SSD, not the array
`domains/` unused VM disks	~170 G	MetaTrader4 + Macinabox Monterey — being converted to compressed qcow2 (2026-07-06)
`onedrive`	578 G	OneDrive sync copy

Retired Unraid VMs archived offsite (2026-07-06) — so they're not forgotten

Proxmox is now the hypervisor of choice; new VMs go on the pve1/pve2/pve3 cluster, not Unraid. The two dormant Unraid VMs were compressed to qcow2 (MetaTrader4 166 G→55 G, Macinabox Monterey 86 G→66 G), then archived encrypted to the Hetzner Storage Box at hetzner-crypt:vm-archive/holly/ and removed from /mnt/user/domains. docker1.img (250 G orphaned docker image) was also removed. To restore a VM: rclone copy hetzner-crypt:vm-archive/holly/<name> <dest> (crypt creds in proxmox/backup/.env), then attach the qcow2 to a new Proxmox VM.

Array Usage (2026-07-03)¶

Disk	Size	Used	Avail	Use%
disk1	3.7T	3.5T	190G	95%
disk2	3.7T	3.5T	236G	94%
disk3	7.3T	6.1T	1.3T	83%
disk4	3.7T	3.5T	184G	96%
disk5	3.7T	3.4T	336G	92%
disk6	3.7T	3.4T	262G	93%

Shares¶

/mnt/user: ClaireDrive, DVDRips, KiCad, MDHAdmin, Media, appdata, caab, domains, downloads, fbshare, games, lancache, onedrive, system, tdarrcache

/mnt/user/media/
├── movies/
├── tv/
├── music/
├── audiobooks/
└── podcasts/

/mnt/user/downloads/
├── complete/
│   ├── movies/
│   ├── tv/
│   └── music/
├── incomplete/
└── watch/

Storage Best Practices¶

Cache Usage

Prefer: Data written to cache first, then moved to array (good for appdata)
Yes: Always on cache (fast but no parity protection)
No: Direct to array (slower but parity protected)
Only: Cache only, never moves to array

Parity Considerations

Parity disk(s) must be equal to or larger than the largest data disk
Two parity disks recommended for arrays with 6+ drives
Monthly parity checks help detect drive issues early
Holly currently runs zero parity by deliberate choice (see the status note above) — all six drives are data disks