Panama/docs/superpowers/plans/2026-08-17-screen-intelligence.md

Screen Intelligence Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add polished local OCR and code recognition to Panama's existing capture experience.

Architecture: Capture keeps ownership of geometry and delegates Read captures to a focused ScreenIntelligence singleton. A deterministic helper emits JSON from local Tesseract and ZBar, while a transient Quickshell result surface owns presentation and explicit follow-up actions.

Tech Stack: Quickshell 0.3.0, Qt 6 QML, Hyprland 0.56 Lua configuration, Bash, Tesseract 5, ZBar, grim, wl-clipboard

Spec: docs/superpowers/specs/2026-08-17-screen-intelligence-design.md

Global Constraints

• Recognition is local until Search, Translate, or Open is explicitly selected.
• Reuse the existing Screen, Window, and Selection picker.
• Use Tokyo Night Moon and Prism tokens; no component-local colors.
• Do not add idle animation or polling.
• Temporary screen captures must be removed when dismissed or replaced.
• Missing recognition packages must degrade into clear guidance, not shell errors.

---

Task 1: Recognition boundary

Files:

• Create: config/dot/quickshell/scripts/screen-intelligence
• Create: tests/quickshell/screen-intelligence-helper-contract.sh
• Modify: setup/packages/hyprland-packages

Interfaces:

• Produces: screen-intelligence probe and screen-intelligence analyze-file IMAGE, each emitting one JSON object.

• Consumes: tesseract, zbarimg, and jq.

• Write a contract that generates a high-contrast PANAMA SCREEN INTELLIGENCE fixture and expects that literal text from analyze-file.

• Run it and verify it fails because the helper does not exist.

• Implement strict argument validation, readiness probing, OCR, code recognition, JSON encoding, and user-safe errors.

• Add tesseract, tesseract-langpack-eng, and zbar to the Hyprland package set.

• Run the helper contract to green with the real engine.

Task 2: Recognition state and actions

Files:

• Create: config/dot/quickshell/services/ScreenIntelligence.qml
• Test: tests/quickshell/screen-intelligence-contract.sh

Interfaces:

• Produces: readiness properties, phase, text, detected-code data, analyzeRegion, analyzeFile, copy/search/translate/open actions, and close.

• Consumes: the Task 1 helper, grim, wl-copy, xdg-open, and Quickshell.cachePath.

• Write a live IPC contract that expects the target, readiness JSON, fixture analysis, and cleanup.

• Run it and verify the IPC target is missing.

• Implement bounded process state, JSON parsing, explicit network actions, and temporary-file cleanup.

• Wire a typed diagnostics and command IPC target in shell.qml.

• Run the live contract to green.

Task 3: Capture integration and result sheet

Files:

• Modify: config/dot/quickshell/services/Capture.qml
• Modify: config/dot/quickshell/modules/capture/CaptureBar.qml
• Create: config/dot/quickshell/modules/capture/IntelligenceResult.qml
• Modify: config/dot/quickshell/shell.qml

Interfaces:

• Produces: Screenshot, Record, and Read action selection plus the transient result overlay.

• Consumes: Task 2's analyzeRegion and state/action API.

• Extend the live contract to open Selection + Read mode and inspect the exposed mode state.

• Run it and verify the Read state is absent.

• Add mutually exclusive action selection and defer recognition until the picker unmaps.

• Build processing, success, empty, code, and unavailable result states with keyboard dismissal.

• Restart Quickshell and inspect fresh logs for construction errors or warnings.

Task 4: Settings, shortcut, launcher, and documentation

Files:

• Create: config/dot/quickshell/modules/settings/ScreenIntelligencePage.qml
• Modify: config/dot/quickshell/modules/settings/SettingsSidebar.qml
• Modify: config/dot/quickshell/modules/settings/SettingsShell.qml
• Modify: config/dot/quickshell/modules/settings/qmldir
• Modify: config/dot/quickshell/services/ShellState.qml
• Modify: config/dot/hypr/keybinds.lua
• Modify: config/dot/hypr/rules.lua
• Create: config/local/share/applications/panama-screen-intelligence.desktop
• Modify: config/dot/hypr/README.md
• Modify: README.md
• Modify: tests/quickshell/settings-pages-contract.sh

Interfaces:

• Produces: Settings destination screen-intelligence, Super+Shift+S, and a searchable desktop entry.

• Extend Settings and keybinding contracts with the new page and shortcut.

• Run them and verify the route and bind fail.

• Build the readiness/privacy page and add it to Settings navigation.

• Add the keybinding, overlay privacy rule, desktop entry, and documentation.

• Link and validate the desktop entry, reload Hyprland, and run contracts to green.

Task 5: Full-system verification

• Run the helper and live Screen Intelligence contracts.
• Run every Quickshell contract and Hyprland --verify-config.
• Restart the live shell and inspect fresh logs.
• Exercise a real screen-region OCR capture and confirm Copy reaches clipboard history.
• Review the complete diff, commit the Panama snapshot, and push main to origin.