Sync both server snapshots with what the live machines run, and keep their backup tooling

server/home/Media/compact-tv/README.md

@@ -0,0 +1,109 @@
+# compact-tv
+
+Sonarr and Radarr each have exactly four quality profiles. Two of them (**Standard** and
+**Mini**) also have their files shrunk by `compact-tv.py`, which runs every night.
+
+| Profile | Used for | What gets kept |
+|---|---|---|
+| **Mini** | shows I demote hard | TV: 720p at 800 kbps (~0.42 GB per hour). Movies: 720p at 1500 kbps |
+| **Standard** | **every request from other users (Seerr default)**, and anything I demote | TV: 1080p at 1200 kbps (~0.6 GB per hour). Movies: 1080p at 2500 kbps (~1.2 GB per hour) |
+| **HD** | my own library, unchanged | whatever downloads, up to 1080p |
+| **UHD** | only me, through Seerr's 4K servers | 4K |
+
+SD sources keep their resolution, at 600 kbps for TV and 900 kbps for movies. Movies get
+higher rates because dark scenes, grain and action break down first at low bitrates. Nothing is ever
+upscaled. Audio becomes Opus: 96 kbps for stereo, 160 for 5.1, 192 for 7.1. Subtitles,
+chapters and attachments are copied as they are. I picked the rates from sample encodes of
+Shameless S05E06 (VMAF ~91 for Standard, ~86 for Mini).
+
+## How the profiles work
+
+- **Standard and Mini** are copies of HD that upgrade only toward small releases (see below),
+  so Sonarr and Radarr never replace a file with a bigger one.
+- **Small downloads first.** Both profiles prefer small releases when they grab:
+  - The "Small Encode" format (MeGusta, PSA, ELiTE, GalaxyTV/RG, YTS/YIFY) scores +3000.
+  - LQ scores 0. In HD and UHD it scores -10000, which keeps those groups out of HD.
+  - Standard merges HDTV/WEB/Bluray at each resolution, so the format score decides between
+    sources rather than the source type.
+  - Mini merges 720p and 1080p into one group and adds +1000 for 720p.
+  - Many files therefore arrive already small, and the job skips them.
+- **Minimum sizes are shared by every profile.** For 720p I lowered them to 5 MB per minute
+  and for 1080p to 6 MB per minute (8 for Bluray-1080p), so small encodes are not rejected.
+  HD still never takes the LQ groups.
+- **Seerr:**
+  - The "Sonarr - HD" and "Radarr" servers default to Standard. Only my admin account sees
+    the profile picker, so other users always get Standard. I pick HD or UHD per request.
+  - Other users no longer have "Request 4K".
+- **Configarr is opt-in** (`docker compose --profile configarr up configarr`). Its templates
+  would recreate the old TRaSH profiles and size limits.
+- Backups from before all this: `backups/pre-four-profiles-20261028/`. That covers both
+  apps' profiles, custom formats and size definitions, which title was on which profile,
+  and Seerr's settings. `setup-small-profiles.py` reapplies the Standard and Mini settings.
+
+## How the job works
+
+The nightly run (00:30 to 08:00) has two phases.
+
+**1. Search first.** It asks Sonarr and Radarr to search titles on Standard or Mini that it
+hasn't searched before, biggest first: 5 shows and 20 movies a night, so downloads and
+indexer use stay spread out. Standard and Mini upgrade only toward small releases:
+- Every allowed quality sits in one group, so no resolution or source upgrade can happen.
+- An upgrade needs a custom-format score jump of at least 2500, and upgrades stop at 3000.
+  Only gaining "Small Encode" clears that bar.
+
+So a search replaces a big file only with a small encode, never with a big HEVC release.
+After that, the apps' regular RSS checks keep picking up small releases on their own.
+
+**2. Encode what's left.** A title becomes eligible for encoding 7 days after its search,
+and only its files that are still big get encoded. Files that arrive after the title's
+search work differently:
+- This covers new episodes of ongoing shows, and fresh requests.
+- The first night they're seen, they get their own search (one episode or movie at a time).
+- They're encoded only if they're still big 3 days later.
+
+Downloads never wait for any of this, so new episodes and requests land in the library as
+soon as they download. Only encoding waits. For each file, the job encodes to
+`~/.cache/compact-tv/work.mkv` on the NVMe and then checks the result:
+- the duration matches the original
+- the audio and subtitle track counts match
+- the file is smaller
+
+It then renames the new file over the original, keeping the same name with `.mkv`, and asks
+Sonarr or Radarr to rescan. Plex picks the change up through its folder watcher, and the
+episode keeps its Plex ID, so watch history stays. If a small release shows up later, the
+app may still replace an encoded file with it, which costs bandwidth, not GPU.
+
+Other details:
+- Each output carries a `COMPACT_TIER` tag. A file is skipped if re-encoding would save less
+  than 30%. Ruled-out files are cached by size in `state.json`, so they aren't probed every
+  night. Search dates and failures are kept there too.
+- A downloaded HEVC file no bigger than twice the target size counts as done. That is the
+  small release a search brought in (a MeGusta 1080p episode runs about 2.3 Mbps against
+  Standard's 1.2), and it is neither re-encoded nor searched again. The job's own tagged
+  outputs are exempt, so Standard to Mini still works.
+- **Skipped:** HDR and Dolby Vision sources (they need tone mapping), subtitle tracks ffprobe
+  can't identify, and files that have failed twice.
+- Speed is about 0.19 hours of GPU per hour of 1080p video (NVENC is the bottleneck). The job
+  stops starting new encodes at 08:00.
+- `cost-report.py` estimates, per title, the GPU hours the job would need against the size of
+  a fresh small download.
+- A `PAUSE` file in this folder makes the nightly run do nothing.
+
+## Going back up
+
+Moving down works: Standard to Mini re-encodes the Standard copy. Moving up does not
+restore quality. To get quality back, I set the title to HD and search, which redownloads
+it at full size.
+
+## Running it
+
+    ./compact-tv.py --dry-run                      # what tonight would do
+    ./compact-tv.py --search-all --limit 0         # search every unsearched title now, encode nothing
+    touch PAUSE                                    # skip nightly runs until PAUSE is deleted
+    ./compact-tv.py --series "Shameless (US)" --limit 1
+    systemctl status compact-tv.timer compact-tv.service
+    tail -f compact-tv.log
+
+The installed units are copies of `compact-tv.service` and `compact-tv.timer` from this
+folder. `ExecStart` goes through `/usr/bin/python3` because SELinux won't let systemd
+execute a file labeled as home content.

server/home/Media/compact-tv/compact-tv.py

@@ -0,0 +1,348 @@
+#!/usr/bin/env python3
+"""Shrink shows and movies that Sonarr/Radarr have on the "Standard" or "Mini" profile.
+
+Demoting something is just picking one of those profiles. Each night this job first asks
+Sonarr/Radarr to search a few titles it hasn't searched before. Both profiles upgrade only
+toward small releases, so the search swaps whatever big files have a small release for
+bandwidth instead of GPU. A week after a title's search, the job re-encodes whatever is
+still big to HEVC on the GPU (NVENC) at the tier's bitrate, verifies the result, swaps it in
+place of the original, and has Sonarr/Radarr rescan. Plex and Jellyfin pick up changes
+through their own folder watchers.
+
+Moving something back up means switching it to HD and searching, which redownloads it at
+full size. See README.md.
+
+Usage:
+  compact-tv.py [--until HH:MM] [--series TITLE] [--limit N] [--dry-run]
+
+--series matches a show or movie title. Runs as gib (files are gib-owned) from
+compact-tv.service, nightly.
+"""
+import argparse, datetime, fcntl, json, os, pathlib, re, shutil, subprocess, sys, time, urllib.request
+
+HERE = pathlib.Path(__file__).resolve().parent
+STATE = HERE / "state.json"
+LOG = HERE / "compact-tv.log"
+WORK = pathlib.Path.home() / ".cache" / "compact-tv"   # NVMe, so half-done encodes never land in a ZFS snapshot
+VOLUMES = pathlib.Path.home() / "Server/Media/volumes"
+
+# Per app: API port, container path -> host path, and how to list one title's files.
+APPS = {
+    "sonarr": {"port": 8989, "paths": ("/tv/", "/home/gib/Public/TV/"), "titles": "/series",
+               "files": lambda call, t: call(f"/episodefile?seriesId={t['id']}"),
+               "rescan": lambda t: {"name": "RescanSeries", "seriesId": t["id"]}},
+    "radarr": {"port": 7878, "paths": ("/movies/", "/home/gib/Public/Movies/"), "titles": "/movie",
+               "files": lambda call, t: [t["movieFile"]] if t.get("movieFile") else [],
+               "rescan": lambda t: {"name": "RescanMovie", "movieId": t["id"]}},
+}
+
+# Video bitrate (kbps) per output class, and the widest output each tier allows.
+# TV floors come from watching samples: 1200 kbps at 1080p, 800 at 720p, 600 at SD.
+# Movies get roughly double: dark scenes, grain and action break down first at low rates,
+# and a 2-hour movie at 2500 kbps is still ~2.3 GB (a quarter of a typical 1080p file).
+# That also leaves already-small releases (YTS-style, ~1.5-2 GB) under the 30% bar.
+# Nothing is ever upscaled: a 720p or SD source keeps its size and uses that class's rate.
+TIERS = {
+    "Standard": {"max_width": 1920, "kbps": {"sonarr": {"1080": 1200, "720": 1200, "sd": 600},
+                                             "radarr": {"1080": 2500, "720": 1800, "sd": 900}}},
+    "Mini":     {"max_width": 1280, "kbps": {"sonarr": {"1080": 800,  "720": 800,  "sd": 600},
+                                             "radarr": {"1080": 1500, "720": 1500, "sd": 900}}},
+}
+MIN_SAVING = 0.30            # leave a file alone unless re-encoding saves at least this much
+MAX_FAILURES = 2             # after this many failed attempts a file is skipped for good
+
+# Search first, encode later. Standard/Mini upgrade only toward small releases (see
+# setup-small-profiles.py), so asking the *arr to search a title swaps whatever big files
+# have a small release, for bandwidth instead of GPU. Each night searches a few titles not
+# searched before, biggest first, so downloads and indexer use stay spread out. A title's
+# files are only encoded once its search is SEARCH_GRACE_DAYS old.
+#
+# Files that arrive after their title's search (new episodes, fresh requests) were never
+# searched for a small copy, so the job searches that episode or movie on its own first,
+# and encodes it only if it is still big FILE_SEARCH_GRACE_DAYS later. Downloads are never
+# delayed by any of this; only encoding waits.
+SEARCHES_PER_NIGHT = {"sonarr": 5, "radarr": 20}
+SEARCH_GRACE_DAYS = 7
+FILE_SEARCH_GRACE_DAYS = 3        # small encodes usually appear within hours to a day; 3 leaves room
+HDR_TRANSFERS = {"smpte2084", "arib-std-b67"}
+
+
+def log(msg):
+    line = f"{datetime.datetime.now():%Y-%m-%d %H:%M:%S}  {msg}"
+    print(line, flush=True)
+    with open(LOG, "a") as f:
+        f.write(line + "\n")
+
+
+# ── Sonarr / Radarr ─────────────────────────────────────────────────────────
+def arr(app):
+    """Return a call(path, body=None) for the app's v3 API."""
+    key = re.search(r"<ApiKey>([^<]+)", (VOLUMES / app / "config.xml").read_text()).group(1)
+    ip = subprocess.run(["docker", "inspect", app, "--format",
+                         "{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}"],
+                        capture_output=True, text=True, check=True).stdout.split()[0]
+    base = f"http://{ip}:{APPS[app]['port']}/api/v3"
+
+    def call(path, body=None):
+        req = urllib.request.Request(base + path, headers={"X-Api-Key": key, "Content-Type": "application/json"},
+                                     data=json.dumps(body).encode() if body is not None else None)
+        with urllib.request.urlopen(req, timeout=60) as r:
+            return json.load(r)
+    return call
+
+
+# ── probing and planning ────────────────────────────────────────────────────
+def probe(path):
+    out = subprocess.run(["ffprobe", "-v", "error", "-print_format", "json", "-show_format", "-show_streams", str(path)],
+                         capture_output=True, text=True)
+    return json.loads(out.stdout) if out.returncode == 0 else None
+
+
+def subtitles(info):
+    """Subtitle tracks ffmpeg can carry over. Tracks ffprobe can't identify are dropped."""
+    return [s for s in info["streams"] if s["codec_type"] == "subtitle" and s.get("codec_name")]
+
+
+def audio_kbps(channels):
+    return 96 if channels <= 2 else 160 if channels <= 6 else 192
+
+
+def plan(info, tier, app):
+    """Return (ffmpeg args, estimated output bytes) for one file, or (None, reason) to skip it."""
+    t = TIERS[tier]
+    video = next((s for s in info["streams"] if s["codec_type"] == "video"
+                  and not s.get("disposition", {}).get("attached_pic")), None)
+    if not video:
+        return None, "no video stream"
+    if video.get("color_transfer") in HDR_TRANSFERS or any("Dolby Vision" in str(sd) for sd in video.get("side_data_list", [])):
+        return None, "HDR/Dolby Vision source (not handled yet)"
+    if (info["format"].get("tags") or {}).get("COMPACT_TIER") == tier:
+        return None, f"already {tier}"
+
+    src_w = int(video["width"])
+    out_w = min(src_w, t["max_width"])
+    cls = "1080" if out_w >= 1700 else "720" if out_w >= 1200 else "sd"
+    vkbps = t["kbps"][app][cls]
+    duration = float(info["format"]["duration"])
+    audios = [s for s in info["streams"] if s["codec_type"] == "audio"]
+    est = int((vkbps + sum(audio_kbps(int(a.get("channels") or 2)) for a in audios)) * 1000 / 8 * duration * 1.03)
+    if int(info["format"]["size"]) * (1 - MIN_SAVING) < est:
+        return None, f"already small ({int(info['format']['size']) / 1e9 * 3600 / duration:.2f} GB/h)"
+    # A downloaded HEVC file within 2x the target is a small release that already replaced a
+    # big one. Re-encoding it would cost GPU and quality for little space. Our own outputs
+    # (tagged) still go through, so moving a title from Standard to Mini works.
+    tags = info["format"].get("tags") or {}
+    if video.get("codec_name") == "hevc" and "COMPACT_TIER" not in tags and int(info["format"]["size"]) <= 2 * est:
+        return None, f"small HEVC release ({int(info['format']['size']) / 1e9 * 3600 / duration:.2f} GB/h)"
+
+    vf = []
+    if video.get("field_order") not in (None, "progressive", "unknown"):
+        vf.append("bwdif")
+    if out_w < src_w:
+        vf.append(f"scale={out_w}:-2:flags=lanczos")
+    args = ["-map", f"0:{video['index']}"]
+    if vf:
+        args += ["-filter:v", ",".join(vf)]
+    args += ["-c:v", "hevc_nvenc", "-preset", "p7", "-tune", "hq", "-profile:v", "main10", "-pix_fmt", "p010le",
+             "-rc", "vbr", "-multipass", "fullres", "-spatial-aq", "1", "-temporal-aq", "1", "-rc-lookahead", "32",
+             "-bf", "4", "-b_ref_mode", "middle",
+             "-b:v", f"{vkbps}k", "-maxrate", f"{vkbps * 2}k", "-bufsize", f"{vkbps * 4}k"]
+    for i, a in enumerate(audios):
+        ch = int(a.get("channels") or 2)
+        layout = {1: "mono", 2: "stereo", 6: "5.1", 8: "7.1"}.get(ch)
+        args += ["-map", f"0:{a['index']}", f"-c:a:{i}", "libopus", f"-b:a:{i}", f"{audio_kbps(ch)}k",
+                 f"-filter:a:{i}", f"aformat=channel_layouts={layout or 'stereo'}"]
+    args += ["-mapping_family", "1"]
+    for i, s in enumerate(subtitles(info)):
+        args += ["-map", f"0:{s['index']}", f"-c:s:{i}", "srt" if s["codec_name"] == "mov_text" else "copy"]
+    args += ["-map", "0:t?", "-c:t", "copy", "-metadata", f"COMPACT_TIER={tier}"]
+    return args, est
+
+
+# ── encode, verify, swap ────────────────────────────────────────────────────
+def encode(src, args, tier):
+    WORK.mkdir(parents=True, exist_ok=True)
+    tmp = WORK / "work.mkv"
+    tmp.unlink(missing_ok=True)
+    cmd = ["ffmpeg", "-hide_banner", "-loglevel", "error", "-nostdin", "-y", "-hwaccel", "cuda", "-i", str(src),
+           *args, "-metadata", f"COMPACT_SOURCE={src.name} ({src.stat().st_size} bytes)", str(tmp)]
+    r = subprocess.run(cmd, capture_output=True, text=True)
+    if r.returncode != 0:
+        tmp.unlink(missing_ok=True)
+        raise RuntimeError("ffmpeg failed: " + r.stderr.strip()[-300:])
+    return tmp
+
+
+def verify(src_info, tmp):
+    new = probe(tmp)
+    if not new:
+        raise RuntimeError("output does not probe")
+    a, b = float(src_info["format"]["duration"]), float(new["format"]["duration"])
+    if abs(a - b) > max(2.0, a * 0.005):
+        raise RuntimeError(f"duration mismatch {a:.1f}s vs {b:.1f}s")
+    counts = lambda i: (sum(1 for s in i["streams"] if s["codec_type"] == "audio"), len(subtitles(i)))
+    if counts(src_info) != counts(new):
+        raise RuntimeError(f"audio/subtitle track counts {counts(src_info)} -> {counts(new)}")
+    if int(new["format"]["size"]) >= int(src_info["format"]["size"]):
+        raise RuntimeError("output is not smaller")
+    return int(new["format"]["size"])
+
+
+def swap(src, tmp):
+    """Copy the encode next to the original, then rename it over the original in one step."""
+    final = src.with_suffix(".mkv")
+    staged = src.with_name(f".{final.name}.compact-tmp")
+    shutil.copyfile(tmp, staged)
+    os.replace(staged, final)
+    if final != src:
+        src.unlink()
+    tmp.unlink(missing_ok=True)
+    return final
+
+
+# ── main ────────────────────────────────────────────────────────────────────
+def main():
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--until", help="stop starting new encodes after this local time (HH:MM)")
+    ap.add_argument("--series", help="only this series title")
+    ap.add_argument("--limit", type=int, help="encode at most N files")
+    ap.add_argument("--dry-run", action="store_true", help="plan only, change nothing")
+    ap.add_argument("--search-all", action="store_true",
+                    help="search every not-yet-searched title now instead of a few per night")
+    opt = ap.parse_args()
+
+    if (HERE / "PAUSE").exists() and not opt.dry_run:
+        log("PAUSE file present, not encoding anything (delete it to resume)")
+        return
+
+    lock = open(HERE / ".lock", "w")
+    try:
+        fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
+    except BlockingIOError:
+        sys.exit("another compact-tv run is active")
+
+    deadline = None
+    if opt.until:
+        h, m = map(int, opt.until.split(":"))
+        now = datetime.datetime.now()
+        deadline = now.replace(hour=h, minute=m, second=0, microsecond=0)
+        if deadline <= now:
+            deadline += datetime.timedelta(days=1)
+
+    state = json.loads(STATE.read_text()) if STATE.exists() else {}
+    for k in ("failures", "searched", "skip"):
+        state.setdefault(k, {})
+    save = lambda: STATE.write_text(json.dumps(state, indent=1))
+    today = datetime.date.today()
+    work = []                                  # (app, call, title, tier); TV first, then movies
+    for app, a in APPS.items():
+        call = arr(app)
+        profiles = {p["id"]: p["name"] for p in call("/qualityprofile")}
+        work += [(app, call, t, profiles[t["qualityProfileId"]]) for t in call(a["titles"])
+                 if profiles.get(t["qualityProfileId"]) in TIERS and (not opt.series or t["title"] == opt.series)]
+    log(f"=== run start{' [dry run]' if opt.dry_run else ''}: {len(work)} title(s) on "
+        f"{'/'.join(TIERS)}{f', until {deadline:%H:%M}' if deadline else ''} ===")
+
+    # ── 1. search: ask the *arr for small replacements before spending GPU ──
+    size_of = lambda t: (t.get("statistics") or {}).get("sizeOnDisk") or t.get("sizeOnDisk") or 0
+    for app in APPS:
+        fresh = sorted((w for w in work if w[0] == app and f"{app}:{w[2]['id']}" not in state["searched"]
+                        and size_of(w[2]) > 0), key=lambda w: -size_of(w[2]))
+        fresh = fresh if opt.search_all else fresh[:SEARCHES_PER_NIGHT[app]]
+        for _, call, t, tier in fresh:
+            log(f"  {'would search' if opt.dry_run else 'search'} {t['title']} ({size_of(t) / 1e9:.0f} GB, {tier})")
+        if fresh and not opt.dry_run:
+            call = fresh[0][1]
+            if app == "sonarr":
+                for _, _, t, _ in fresh:
+                    call("/command", {"name": "SeriesSearch", "seriesId": t["id"]})
+            else:
+                call("/command", {"name": "MoviesSearch", "movieIds": [t["id"] for _, _, t, _ in fresh]})
+            for _, _, t, _ in fresh:
+                state["searched"][f"{app}:{t['id']}"] = today.isoformat()
+            save()
+
+    # ── 2. encode what is still big after its search ────────────────────────
+    age = lambda iso: (today - datetime.date.fromisoformat(iso)).days
+
+    def ready(app, ef, title_searched, to_search):
+        """True if this big file has had its chance to be replaced by a small download."""
+        if (ef.get("dateAdded") or "")[:10] <= title_searched:   # was on disk when the title was searched
+            return age(title_searched) >= SEARCH_GRACE_DAYS
+        key = f"{app}:file:{ef['id']}"                            # arrived later: search it on its own first
+        if key not in state["searched"]:
+            to_search.append(ef)
+            return False
+        return age(state["searched"][key]) >= FILE_SEARCH_GRACE_DAYS
+
+    def search_files(app, call, show, files):
+        if opt.dry_run:
+            log(f"  would search {len(files)} newer file(s) of {show['title']}"); return
+        if app == "sonarr":
+            ids = {f["id"] for f in files}
+            eps = [e["id"] for e in call(f"/episode?seriesId={show['id']}") if e.get("episodeFileId") in ids]
+            call("/command", {"name": "EpisodeSearch", "episodeIds": eps})
+        else:
+            call("/command", {"name": "MoviesSearch", "movieIds": [show["id"]]})
+        for f in files:
+            state["searched"][f"{app}:file:{f['id']}"] = today.isoformat()
+        log(f"  search {len(files)} newer file(s) of {show['title']} before encoding"); save()
+
+    done = saved = 0
+    for app, call, show, tier in work:
+        title_searched = state["searched"].get(f"{app}:{show['id']}")
+        if not title_searched:
+            continue                          # its title search comes first, on a later night
+        container, host = APPS[app]["paths"]
+        files = sorted(APPS[app]["files"](call, show), key=lambda f: f["path"])
+        changed, to_search = False, []
+        for ef in files:
+            if deadline and datetime.datetime.now() >= deadline:
+                log("deadline reached, stopping"); break
+            if opt.limit is not None and done >= opt.limit:
+                break
+            src = pathlib.Path(ef["path"].replace(container, host, 1))
+            if state["failures"].get(str(src), 0) >= MAX_FAILURES or not src.exists():
+                continue
+            # Files already ruled out at this tier and size need no re-probe (most of them, most nights).
+            seen = f"{tier}:{src.stat().st_size}"
+            if state["skip"].get(str(src)) == seen:
+                continue
+            info = probe(src)
+            if not info:
+                log(f"  SKIP {src.name}: does not probe"); continue
+            args, est = plan(info, tier, app)
+            if args is None:
+                state["skip"][str(src)] = seen  # reason in `est`; quiet, since most nights most files are done
+                continue
+            if not ready(app, ef, title_searched, to_search):
+                continue
+            old = int(info["format"]["size"])
+            if opt.dry_run:
+                log(f"  would encode {src.name}: {old / 1e6:.0f} MB -> ~{est / 1e6:.0f} MB ({tier})")
+                done += 1; continue
+            t0 = time.monotonic()
+            try:
+                tmp = encode(src, args, tier)
+                new = verify(info, tmp)
+                final = swap(src, tmp)
+            except Exception as e:
+                state["failures"][str(src)] = state["failures"].get(str(src), 0) + 1
+                save()
+                log(f"  FAIL {src.name}: {e}"); continue
+            done += 1; saved += old - new; changed = True
+            log(f"  {final.name}: {old / 1e6:.0f} MB -> {new / 1e6:.0f} MB "
+                f"(-{(1 - new / old) * 100:.0f}%, {time.monotonic() - t0:.0f}s, {tier})")
+        if to_search:
+            search_files(app, call, show, to_search)
+        if changed:
+            call("/command", APPS[app]["rescan"](show))
+        if (deadline and datetime.datetime.now() >= deadline) or (opt.limit is not None and done >= opt.limit):
+            break
+    if not opt.dry_run:
+        save()
+    log(f"=== run end: {done} file(s){' planned' if opt.dry_run else ''}, saved {saved / 1e9:.1f} GB ===")
+
+
+if __name__ == "__main__":
+    main()

server/home/Media/compact-tv/compact-tv.service

@@ -0,0 +1,17 @@
+# Installed to /etc/systemd/system/. Runs as gib because the TV files are gib-owned,
+# so the job needs no root. A system unit (rather than a user unit) because gib has
+# no lingering session on this headless box. ExecStart goes through python3 because
+# SELinux will not let systemd execute a file labeled as home content (user_home_t).
+[Unit]
+Description=Re-encode TV shows on Sonarr's Compact/Supercompact profiles
+Documentation=file:///home/gib/Server/Media/compact-tv/README.md
+After=docker.service
+Requires=docker.service
+
+[Service]
+Type=oneshot
+User=gib
+Group=gib
+ExecStart=/usr/bin/python3 /home/gib/Server/Media/compact-tv/compact-tv.py --until 08:00
+Nice=15
+IOSchedulingClass=idle

server/home/Media/compact-tv/compact-tv.timer

@@ -0,0 +1,11 @@
+# Starts after the 00:00 hour and stops starting new encodes at 08:00 (see --until in
+# the service), so Plex and Frigate keep the GPU's encoder during the day.
+[Unit]
+Description=Nightly TV re-encode for Compact/Supercompact shows
+
+[Timer]
+OnCalendar=*-*-* 00:30:00
+Persistent=false
+
+[Install]
+WantedBy=timers.target

server/home/Media/compact-tv/cost-report.py

@@ -0,0 +1,65 @@
+#!/usr/bin/env python3
+"""Per-title cost of shrinking everything on Standard/Mini: re-encode vs redownload.
+
+For each show or movie on those profiles, probes every file with compact-tv's own planner
+and reports what the nightly job would do (encode, skip as already small, skip as HDR),
+the space it would save, and the GPU time it would take. Alongside that it estimates what a
+fresh small download would cost instead, using typical small-encode sizes.
+
+GPU time is measured, not guessed: the Shameless pilot took 669 s for 57.5 minutes of
+1080p (about 0.19 h of GPU per hour of video). 4K sources decode slower; the 1.5x for them
+is an estimate until the first 4K night is logged.
+
+Usage: cost-report.py > report.tsv     (also prints a summary to stderr)
+"""
+import importlib.util, json, pathlib, sys
+
+HERE = pathlib.Path(__file__).resolve().parent
+spec = importlib.util.spec_from_file_location("ct", HERE / "compact-tv.py")
+ct = importlib.util.module_from_spec(spec); spec.loader.exec_module(ct)
+
+GPU_H_PER_H = 669 / 3450                 # pilot: hours of GPU per hour of 1080p video
+UHD_FACTOR = 1.5                         # estimate for 2160p sources
+SMALL_GB_PER_H = {"Standard": 0.85, "Mini": 0.57}   # MeGusta-class 1080p / 720p HEVC, measured on S.W.A.T.
+
+rows = []
+for app, a in ct.APPS.items():
+    call = ct.arr(app)
+    profiles = {p["id"]: p["name"] for p in call("/qualityprofile")}
+    container, host = a["paths"]
+    for t in call(a["titles"]):
+        tier = profiles.get(t["qualityProfileId"])
+        if tier not in ct.TIERS:
+            continue
+        r = {"app": app, "title": t["title"], "tier": tier, "files": 0, "hours": 0.0, "size": 0,
+             "enc_files": 0, "enc_size": 0, "enc_est": 0, "gpu_h": 0.0, "hdr_size": 0, "small_size": 0}
+        for f in a["files"](call, t):
+            p = pathlib.Path(f["path"].replace(container, host, 1))
+            info = ct.probe(p) if p.exists() else None
+            if not info:
+                continue
+            size, dur = int(info["format"]["size"]), float(info["format"]["duration"]) / 3600
+            r["files"] += 1; r["hours"] += dur; r["size"] += size
+            args, why = ct.plan(info, tier, app)
+            if args:
+                w = max((s.get("width") or 0) for s in info["streams"] if s["codec_type"] == "video")
+                r["enc_files"] += 1; r["enc_size"] += size; r["enc_est"] += why
+                r["gpu_h"] += dur * GPU_H_PER_H * (UHD_FACTOR if w > 1920 else 1)
+            elif "HDR" in str(why):
+                r["hdr_size"] += size
+            else:
+                r["small_size"] += size
+        if r["files"]:
+            r["redownload_gb"] = round(r["hours"] * SMALL_GB_PER_H[tier], 1)
+            rows.append(r)
+
+rows.sort(key=lambda r: -(r["enc_size"] - r["enc_est"]))
+cols = ["app", "title", "tier", "files", "hours", "size", "enc_files", "enc_size", "enc_est", "gpu_h", "hdr_size",
+        "small_size", "redownload_gb"]
+print("\t".join(cols))
+for r in rows:
+    print("\t".join(str(round(r[c], 2) if isinstance(r[c], float) else r[c]) for c in cols))
+tot = lambda k: sum(r[k] for r in rows)
+print(f"{len(rows)} titles; encode {tot('enc_files')} files {tot('enc_size')/1e12:.2f} TB -> ~{tot('enc_est')/1e12:.2f} TB, "
+      f"{tot('gpu_h'):.0f} GPU hours; HDR skipped {tot('hdr_size')/1e12:.2f} TB; already small {tot('small_size')/1e12:.2f} TB",
+      file=sys.stderr)

server/home/Media/compact-tv/setup-small-profiles.py

@@ -0,0 +1,99 @@
+#!/usr/bin/env python3
+"""Make Standard and Mini prefer small releases, and upgrade only toward small ones.
+
+Run once per app; it is idempotent, so re-running updates everything in place.
+
+- Adds "Small Encode" (release groups known for small HEVC/x264 encodes) and "720p" /
+  "1080p" resolution formats. They score 0 in every profile except Standard and Mini.
+- HD and UHD are otherwise untouched: they keep LQ at -10000, so they never pick these.
+- Standard and Mini:
+  - All allowed qualities share ONE group, so no "better resolution" or "better source"
+    upgrade can ever download a big file. Only the custom-format score decides.
+  - Scores: Small Encode +3000, x265 +2000 (so first grabs still prefer HEVC), LQ 0,
+    release-tier formats 0 (one big release must never "beat" another big one).
+    Standard adds +500 for 1080p and +250 for 720p; Mini adds +1000 for 720p and +500 for
+    1080p. Those preferences stay below the upgrade threshold, so they never cause churn.
+  - Upgrades are on, but an upgrade needs a score jump of at least 2500, and upgrading
+    stops at 3000. In practice only gaining "Small Encode" clears that bar: a big H.264
+    file gets replaced by a small encode, but never by a big HEVC one.
+
+Usage: setup-small-profiles.py <sonarr|radarr>
+"""
+import json, re, subprocess, sys, urllib.request, pathlib
+
+APP = sys.argv[1]
+cfg = pathlib.Path.home() / f"Server/Media/volumes/{APP}/config.xml"
+key = re.search(r"<ApiKey>([^<]+)", cfg.read_text()).group(1)
+ip = subprocess.run(["docker", "inspect", APP, "--format", "{{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}"],
+                    capture_output=True, text=True, check=True).stdout.split()[0]
+base = f"http://{ip}:{8989 if APP == 'sonarr' else 7878}/api/v3"
+
+def call(path, body=None, method=None):
+    req = urllib.request.Request(base + path, method=method, headers={"X-Api-Key": key, "Content-Type": "application/json"},
+                                 data=json.dumps(body).encode() if body is not None else None)
+    with urllib.request.urlopen(req, timeout=60) as r:
+        return json.load(r)
+
+GROUPS = r"^(MeGusta|PSA|ELiTE|GalaxyTV|GalaxyRG|GalaxyRG265|YIFY|YTS(\.(MX|LT|AG))?)$"
+resolution = lambda r: [{"name": f"{r}p", "implementation": "ResolutionSpecification",
+                         "negate": False, "required": True, "fields": [{"name": "value", "value": r}]}]
+WANTED = {
+    "Small Encode": [{"name": "Small encode groups", "implementation": "ReleaseGroupSpecification",
+                      "negate": False, "required": False, "fields": [{"name": "value", "value": GROUPS}]}],
+    "720p": resolution(720),
+    "1080p": resolution(1080),
+}
+TIERS = {
+    "Standard": {"Small Encode": 3000, "x265 (HD)": 2000, "LQ": 0, "LQ (Release Title)": 0, "1080p": 500, "720p": 250},
+    "Mini":     {"Small Encode": 3000, "x265 (HD)": 2000, "LQ": 0, "LQ (Release Title)": 0, "720p": 1000, "1080p": 500},
+}
+RELEASE_TIER = re.compile(r"Tier \d+|WEB Scene")
+MIN_UPGRADE_JUMP, UPGRADE_UNTIL = 2500, 3000
+
+cfs = {c["name"]: c for c in call("/customformat")}
+for name, specs in WANTED.items():
+    body = {"name": name, "includeCustomFormatWhenRenaming": False, "specifications": specs}
+    if name in cfs:
+        body["id"] = cfs[name]["id"]; call(f"/customformat/{body['id']}", body, "PUT")
+    else:
+        call("/customformat", body)
+cfs = {c["name"]: c["id"] for c in call("/customformat")}
+
+
+def flatten(items):
+    """Every quality in the profile with whether it is allowed, groups expanded, lowest rank first."""
+    for it in items:
+        for sub in (it["items"] or [it]):
+            yield sub["quality"], it["allowed"]
+
+
+def one_group(profile):
+    """Put every allowed quality into a single top-ranked group; leave disallowed ones as they are."""
+    flat = list(flatten(profile["items"]))
+    group = {"id": 1100, "name": "Any allowed", "allowed": True,
+             "items": [{"quality": q, "items": [], "allowed": True} for q, ok in flat if ok]}
+    profile["items"] = [{"quality": q, "items": [], "allowed": False} for q, ok in flat if not ok] + [group]
+    profile["cutoff"] = group["id"]
+    return profile
+
+
+for prof in call("/qualityprofile"):
+    have = {f["format"] for f in prof["formatItems"]}
+    for name in WANTED:
+        if cfs[name] not in have:
+            prof["formatItems"].append({"format": cfs[name], "name": name, "score": 0})
+    scores = TIERS.get(prof["name"])
+    if scores:
+        prof = one_group(prof)
+        prof.update(upgradeAllowed=True, minUpgradeFormatScore=MIN_UPGRADE_JUMP, cutoffFormatScore=UPGRADE_UNTIL)
+        for f in prof["formatItems"]:
+            if f["name"] in scores:
+                f["score"] = scores[f["name"]]
+            elif RELEASE_TIER.search(f["name"]):
+                f["score"] = 0
+    call(f"/qualityprofile/{prof['id']}", prof, "PUT")
+    if scores:
+        allowed = [q["quality"]["name"] for it in prof["items"] if it["allowed"] for q in it["items"]]
+        print(f"{APP} {prof['name']}: one group of {len(allowed)} qualities ({allowed[0]} .. {allowed[-1]}); "
+              f"upgrades on, jump >= {MIN_UPGRADE_JUMP}, until {UPGRADE_UNTIL}")
+        print("   nonzero scores: " + ", ".join(f"{f['name']}={f['score']}" for f in prof["formatItems"] if f["score"]))

server/home/Media/compose.yml

@@ -427,3 +427,8 @@ services:
     # meant a completed sync was immediately restarted -- a hot loop of ~9
     # full syncs every 3 minutes. `no` lets it run once per `up` and stop.
     restart: "no"
+    # Opt-in only (`docker compose --profile configarr up configarr`). Its templates
+    # create the TRaSH profiles and size limits that were removed on 2026-10-28 in
+    # favour of four hand-managed profiles (Mini/Standard/HD/UHD), so a plain `up`
+    # must never run it. See Media/compact-tv/README.md.
+    profiles: [configarr]