Gabriel Brown
d96863b687
colour -> color, behaviour -> behavior, centre -> center, favourite -> favorite, and about twenty other pairs, applied consistently across comments, docs, error/UI copy, and a handful of QML identifiers that used the British spelling as their actual name: SystemSettings' serialiseValue/serialiseTable/normaliseGradient, Displays' normaliseModes, Wallpaper's normalisePolicy, SettingsBackup's serialiseHomeState, DateTime's ntpSynchronised property, Clipboard's _normalise helper, and ShortcutCapture's cancelled signal (with its onCancelled handler in ShortcutsPage.qml). Every call site and the two tests that assert on the literal source text (settings-ownership and settings-backup-live contracts) were updated in lockstep. Left untouched: config/dot/espanso/match/packages/misspell-en/ is a vendored third-party autocorrect dictionary -- its entries are typo corrections, not our prose, and rewriting them would fight the package's own purpose (and any future re-sync from upstream). The already-American `favorites` property (Home page pinned accessories) was never actually misspelled -- only nearby comments and error strings said "favourites" -- so no data migration was needed there. Claude-Session: https://claude.ai/code/session_01E6TJUAh41HaP25MVHWkhRZ
71 lines
4.3 KiB
Markdown
71 lines
4.3 KiB
Markdown
# Screen Intelligence Design
|
|
|
|
## Purpose
|
|
|
|
Screen Intelligence makes text trapped in pixels feel like normal desktop content. It extends Panama's existing screenshot picker with a local **Read** action for a screen, window, or selection, then presents the result in a focused Prism glass sheet.
|
|
|
|
The feature is deliberately local-first. Screen pixels are captured into Panama's cache and processed by Tesseract and ZBar on the workstation. Nothing is uploaded unless the user explicitly chooses Search, Translate, or Open.
|
|
|
|
## Interaction model
|
|
|
|
- `Super+Shift+S` opens the existing capture picker in Selection + Read mode.
|
|
- `Print` keeps opening the complete picker, where Screenshot, Record, and Read are peers.
|
|
- Screen, Window, and Selection continue to use one shared target-selection model.
|
|
- Committing a Read capture dismisses the picker before taking the final image, then opens a result sheet while recognition runs.
|
|
- `Escape` dismisses the result and deletes its temporary image.
|
|
- The result provides selectable text and explicit Copy, Search, and Translate actions.
|
|
- A detected URL, QR code, or barcode is shown separately. Web URLs can be opened; all detected values can be copied.
|
|
- A searchable desktop entry opens Read mode without requiring the shortcut.
|
|
|
|
## Architecture
|
|
|
|
`services/Capture.qml` remains the owner of screen/window/selection geometry and frozen-frame behavior. It gains a third action state, `intelligenceMode`, and delegates committed Read geometry to `services/ScreenIntelligence.qml` only after the capture overlay has unmapped.
|
|
|
|
`services/ScreenIntelligence.qml` owns recognition state, process lifecycle, result actions, and cleanup. Its public boundary is:
|
|
|
|
- `refresh(): void`
|
|
- `analyzeRegion(geometry: string, outputName: string): void`
|
|
- `analyzeFile(path: string): void`
|
|
- `copyText(): void`
|
|
- `copyCode(): void`
|
|
- `search(): void`
|
|
- `translate(): void`
|
|
- `openDetected(): void`
|
|
- `close(): void`
|
|
|
|
`scripts/screen-intelligence` is the deterministic machine boundary. `probe` reports engine availability. `analyze-file IMAGE` emits one JSON object containing OCR text, detected code data, and a user-safe error. It never evaluates UI-provided shell text.
|
|
|
|
`modules/capture/IntelligenceResult.qml` is a transient overlay, not an application window. It renders a quiet processing state, the selectable result, detected-content card, actions, and actionable failure guidance.
|
|
|
|
## Recognition behavior
|
|
|
|
- OCR uses the installed English Tesseract language pack.
|
|
- Tesseract's automatic page-segmentation mode handles both short snippets and larger document regions.
|
|
- ZBar recognition is additive. A capture can return text, a code, or both.
|
|
- Empty recognition is not treated as a crash; the sheet explains that no readable text or code was found.
|
|
- Missing engines produce a precise install command and keep the rest of the shell healthy.
|
|
- Temporary captures live under `Quickshell.cachePath(...)` and are removed when dismissed or replaced.
|
|
|
|
## Visual language
|
|
|
|
- The result is a centered Prism glass sheet over a lightly dimmed desktop.
|
|
- Tokyo Night Moon tokens are the only colors.
|
|
- Processing uses a static status treatment; there is no spinner, pulse, shimmer, or idle animation.
|
|
- OCR text is readable interface typography in a selectable editor, not a terminal-like dump.
|
|
- The primary action is Copy. Network-leaving actions are visually secondary and clearly labeled.
|
|
|
|
## Settings and discoverability
|
|
|
|
Panama Settings gains a real **Screen Intelligence** page. It shows engine readiness, the `Super+Shift+S` shortcut, the local-processing privacy boundary, and the exact package requirement if recognition is unavailable. The page contains no fake toggles.
|
|
|
|
The shortcut reference, Hyprland README, root README, desktop package list, and desktop entry remain in sync.
|
|
|
|
## Verification
|
|
|
|
- A generated image fixture must be recognized as literal text by the helper.
|
|
- The live IPC contract must expose readiness, open Read mode, and route the Settings page.
|
|
- The capture picker must construct with Screenshot, Record, and Read actions without QML warnings.
|
|
- The result sheet must construct for processing, success, empty, and unavailable states.
|
|
- Hyprland must register `Super+Shift+S` as Screen Intelligence and pass config verification.
|
|
- The complete Quickshell contract suite must remain green.
|