pigway-frame-flow/README.md

# Pigway Frame Flow

Pigway Frame Flow is a lightweight self-hosted platform for programmable smart-frame automation.

独立运行的智能画框服务：中英文 Web UI、深浅色模式、设备驱动抽象、KoKonna 全部已公开接口、图片处理、模板、SQLite 调度与播放列表、临时覆盖恢复和 REST API。

KoKonna is a supported device integration and is not affiliated with this project unless otherwise stated.

## Quick Start / 快速开始

```sh
python3 -m venv .venv
.venv/bin/python -m pip install -r requirements.lock
.venv/bin/python scripts/run_local.py
```

Open http://127.0.0.1:18787. The local demo creates a private `.env` with a random
FRAME_FLOW_TOKEN. Read that token locally in an editor and enter it on the login
screen. Do not paste a KoKonna API Key into Web UI. Demo always forces mock mode
and disables real writes. It leaves data persistent in ./data and ./config.

网页使用账号密码登录。通过 `python scripts/create-account.py --username YOUR_NAME` 创建账号，再启动服务；密码不会写入代码或明文配置。勾选记住登录可保持 30 天，设置页面可修改密码或退出。

## Architecture / 架构

```
providers → templates / renderers → generated image
                                  ↓
Web UI / external REST → DisplayService → FrameDeviceDriver → KokonnaDriver
Scheduler / Playlist ─────────────┘
                   SQLite: operations / history / jobs / overrides / assets
```

Package: pigway_frame_flow. Device composition is in core/registry.py;
vendor-specific code is in drivers/kokonna. One process, one worker, no Redis,
message broker, OpenClaw, Home Assistant or external application dependency.

## Configuration / 配置

Copy `.env.example` to `.env`, set a strong FRAME_FLOW_TOKEN, chmod 600 .env.
- KOKONNA_MOCK=true: independent persistent demo, no cloud calls.
- KOKONNA_WRITE_ENABLED=false: live write protection, mandatory before verification.
- KOKONNA_API_KEY: device key, never stored in DB or returned to Web UI.
- *_FILE: optional Docker Secret equivalent.
- WEB_BIND/WEB_PORT: Compose host bind and port. Internal app binds container network.
- CORS_ALLOWED_ORIGINS: exact comma-separated origins; default empty.
- IMAGE_URL_ALLOWED_HOSTS: exact public HTTPS hosts; default disabled. Redirects denied.
- TZ=Asia/Shanghai; DATA_DIR=/data; CONFIG_DIR=/config.

For multiple devices copy deploy/devices.yml.example into config/devices.yml,
reference a separate environment key per device. Jobs bind a device_id. Layout
currently targets 1200 × 1600; incompatible geometry is refused before writes.

## Docker

```sh
cp .env.example .env
# Configure a token; keep API key empty, mock true, writes false.
mkdir -p data/images data/generated data/cache config
sudo chown -R 10333:10333 data config
./scripts/build-verify.sh
docker compose up -d
python3 scripts/smoke-test.py
```

Image pigway-frame-flow:latest, service/container pigway-frame-flow, AMD64,
Python 3.14.5, pinned runtime dependencies, CJK fonts, non-root UID/GID 10333,
read-only root filesystem, writable /data and /config, no capabilities, no host
network, no Docker socket. Restart unless-stopped. JSON-file logs capped 10m × 3.
Healthcheck uses service /health, not cloud availability. No second log database.

## Synology NAS

Target `/volume2/docker/pigway-frame-flow/`. See [NAS deployment](docs/NAS.md).
Set WEB_BIND to the NAS LAN IP to restrict the published interface, or explicitly
0.0.0.0 when desired. Port 18787 must be free. No router/firewall/reverse proxy
rules are modified. Source and persistent volumes stay in this project directory.
No artificial logs directory is required. Compose creates its own bridge network.

## KoKonna Setup / 真实设备

First deploy Mock and verify persistence. Then set KOKONNA_MOCK=false and fill
KOKONNA_API_KEY through private environment/secret. Keep writes false. /ready checks
configuration/DB/worker/token without a cloud call. Query device and images metadata;
compare model, geometry, timezone, heartbeat, battery, imageId, coverId, sync.
If different, stop and report. **Do not enable writes or perform the single test
until the owner explicitly authorizes it.** Test design: [REAL_DEVICE_TEST.md](docs/REAL_DEVICE_TEST.md).

## Security

Protected /api/v1 capabilities accept an administrator account session, or external
Frame Flow Bearer auth with scope checks.
SQLite stores token hashes. Device keys live only in env/secret and runtime memory.
Browser login uses SQLite accounts with salted scrypt password hashes and revocable,
HttpOnly/SameSite=Strict cookies (30 days when remembered, 12 hours otherwise).
Mutations require a session CSRF header. Logout and password changes revoke sessions.
Passwords and browser sessions are never stored in localStorage. Bootstrap hashes
in private `config/accounts.json` seed new accounts only; restarting does not undo
a password change. Five failed logins per source IP cause a five-minute lockout.
The browser needs no API Token. Bearer authentication remains compatible for
external clients; the KoKonna key is never used for account login.
The LAN deployment uses HTTP; use an explicitly configured HTTPS endpoint if
transport encryption is needed. No reverse proxy or firewall is changed automatically.

## API

OpenAPI 3.x at /openapi.json; Swagger at /docs; stable /api/v1. Display/render/upload
return 202 + operation_id. Poll /api/v1/operations/{id}; completion requires matching
image ID and cloud synced flag. Idempotency-Key retained seven days. X-Request-ID
traces history. UNKNOWN writes block later mutations; read-only reconciliation
and explicit admin acknowledgement release the block, never replay a write.
The older /api/display route is retained and deprecated (same v1 payload).
See [external integration contract](docs/integrations/TICKTAR.md).

## Scheduler / Playback

Cron (5 fields), daily, weekdays, weekends, date and temporary tasks; Shanghai time.
Use source {"type":"restore"} for explicit restore jobs. Unique job/slot and persistent
idempotency prevent duplicate dispatch. Overdue tasks >5 minutes are skipped rather
than unexpectedly overwriting artwork after downtime. State persists across restart.
Overrides use priority then recency; lower priorities wait without uploading.
If another app changes the frame, expired overrides do not overwrite its artwork.
Frame Flow playlists support ordered/random cloud image IDs and >=5-minute intervals;
only one enabled playlist per device. They pause for overrides/unknown writes.
Native cloud slideshow mode/interval are readable; membership/editing are unavailable
in the official API. Avoid having native and Frame Flow playback compete.

## Templates / 图片

Seven JSON layout templates: daily-dashboard, weather, calendar, quote, photo-text,
daily-poetry and system-alert, with color and grayscale profiles. Every template
renders an image, including templates that contain only text.

使用流程：选择模板 → 编辑数据 → 生成预览 → 显示到画框。生成预览只创建本地图片；
保存默认数据只保存配置，两者都不会更新设备。自动化模板任务每次运行重新生成图片。

- **照片与文字 / Photo + Text**：标题、中央照片区域、下方正文；与照片数据源任务的底部水印布局不同。
- **家庭早报 / Daily Dashboard**：日期、天气、事项、一句话与所选照片区域。
- **天气 / Weather**：当前天气与三天预报；天气 JSON 留空时使用配置的 Provider。
- **日历、每日诗词、一句话、系统告警**：将相应信息排版为图片，不自动选择图库照片。

Photos use a local asset ID (`photo_id`); without a selected photo, a placeholder
is rendered. Configure Open-Meteo and location coordinates in **Settings → Weather
source** for real weather. Forecasts are fetched on demand and cached for ten minutes;
unavailable/stale data is labeled explicitly. Weather remains static on the frame
until the next render and display, so use an Automation job for regular updates.
Official KoKonna widgets are a separate integration, not these local templates.
No unreliable lunar date is fabricated.
Contain preserves composition with margins; cover crops centrally; smart is an
edge-saliency heuristic. Always inspect preview; face-aware guarantees are not claimed.
Cloud uploads activate immediately, so gallery-only means **local staging**.

## Development / Testing

```sh
.venv/bin/python -m pip install -e '.[test]'
.venv/bin/python -m pytest -q
.venv/bin/python -m ruff check pigway_frame_flow tests scripts
.venv/bin/python -m ruff format --check pigway_frame_flow tests scripts
.venv/bin/python -m mypy pigway_frame_flow
.venv/bin/python -m pip_audit -r requirements.lock --no-deps --disable-pip
```

Tests mock every cloud request; no real-device writes. scripts/ui-smoke.cjs runs an
isolated headless browser and checks settings.mock before acting. CI builds AMD64,
validates Compose, starts a container, checks health/ready and restarts for persistence.

## Known Limitations / 已知边界

Cloud acknowledgements do not prove physical display pixels. Model fields may be
omitted and are explicitly marked configured. No native slideshow membership API,
no cloud upload-only API, no automatic retries after uncertain writes. Opt-in local retention protects manual uploads, stars and active references; its quota is a cleanup target, not a hard disk limit. Rendering assets are local, limited
to 25M pixels/20 MiB. This release uses a single worker and is not multi-replica.
Gallery reads have no pagination because upstream documents none. Web UI hides
secrets and lazy-loads thumbnails; generating a first thumbnail fetches one original.
Scoped client provisioning and bounded signed webhook delivery are available in Connections. Docker build/NAS
status is recorded honestly in docs/ACCEPTANCE.md.

## Roadmap

Browser OAuth login, AI daily-art scheduling, custom ICS timezones/EXRULE and additional hardware-specific adapters remain pending. MQTT, bounded recurrence, token refresh, transition events and single-image AI generation are optional capabilities. See [optional capabilities and exact limits](docs/EXTENSIONS.md).

## Layout Studio and optional integrations

Bilingual Layout Studio imports 1–12 images, edits bounded regions and renders a collage or poster. Optional OpenAI-compatible AI plans receive text and dimensions only. Connections configures provider credentials, calendars/state, retention, client scopes and webhooks. Photo Sources includes NAS, WebDAV, Google Drive, AliPan and Baidu readers. Authorization is separate from implementation; unconnected providers return explicit errors. See [setup and limitations](docs/EXTENSIONS.md).

## Template editing

Templates provide bilingual form fields, event lists, local photo selection, advanced
JSON, color/grayscale previews, persistent defaults and reset to built-in defaults.
Changing content invalidates the previous preview. Weather uses the labeled Mock
provider until a real provider is configured. Live display controls stay disabled
until real writes are separately approved.

Browser smoke tests use a disposable Mock account via FRAME_FLOW_TEST_USERNAME
and FRAME_FLOW_TEST_PASSWORD; never run the mutation smoke suite against live devices.

## Current content and completed page workflows

Dashboard displays the actual current image and recorded content source. Frame Flow
stores immutable template/data/profile snapshots when it renders an asset and maps
successful upload results to the device/image ID. The current-content endpoint
matches the live device image ID; vendor images with no recorded source remain
explicitly unknown. Switching externally never makes the previous template appear
as the current one. Restore also restores the matching content provenance.

Templates can load the current recorded template or continue editing a generated
local image. They support direct photo import and protected image downloads.
Gallery provides search, sorting, dimensions and downloads. Display confirmation
supports duration, priority and restoration. History includes operation details.
Automation offers content pickers, template profiles, previews and manual execution;
playlists offer gallery image checkboxes. Real writes remain blocked by the server
regardless of the UI controls.

Settings persists weather coordinates/provider and image URL host allowlists in
SQLite. Open-Meteo provides current model weather and a three-day forecast, cached
for ten minutes on demand. Network failures show unavailable/stale data and use a
two-minute backoff, without replacing it with fictitious live data. Default remains
Mock until configured in Settings. Open-Meteo attribution is included in the UI and
weather/dashboard renders. Image URL import is HTTPS-only with explicit hostnames,
public-address validation and pinned DNS; redirects and private addresses are blocked.

Additional v1 endpoints: `/devices/{id}/current-content`, `/assets/{id}/source`,
`/providers/weather`, `/settings/weather`, `/settings/image-url`, `/jobs/{id}`,
`/jobs/{id}/preview`, `/jobs/{id}/run` (all under `/api/v1`). Existing contracts remain
compatible; new endpoints are included in generated OpenAPI and contract tests.

## Daily Poetry / 每日诗词

The daily-poetry template includes 20 public-domain classical Chinese poems, selected deterministically by Shanghai date, or manually. Create a daily job from its editor; new poetry jobs are paused by default until you enable them. Custom title, author and verses are editable. This is an independent Frame Flow content provider, not the KoKonna mini-program subscription. Official public API does not expose mini-program subscription/template settings.

每日诗词支持按上海日期选诗、手选 20 首古典诗词、自定义文本以及每日任务。模板显示操作使用统一上传/显示服务；真实写入须开启 KOKONNA_WRITE_ENABLED，结果未知不会自动重试。官方小程序原有每日诗词订阅不能通过当前公开 API 查询或修改。


### 官方账号连接 / Official account connection

在 Templates → 官方小组件中，由管理员输入 KoKonna 官方账号和密码，查询账号下
的设备后手动关联当前 Frame Flow 设备。此流程不绑定或解绑云端设备，也不改变
图片或订阅。现有 API Key 接入继续负责设备、图库和显示能力。

密码不保存；会话与关联信息保存在 `/config/official-{device_id}.secret`，权限 0600。
容器重启可恢复连接。会话过期需要重新登录；断开连接会清除本地会话，不撤销
官方服务器上的其他登录。局域网访问沿用现有部署方式。

Official account connection is admin-only and separate from API Key access.
Login, device discovery, local association and disconnect are available through
`/api/v1/devices/{device_id}/official/connection`. Passwords are never persisted;
server-side sessions are stored as restricted secret files. No official widget
mutation or other extra RPC write is enabled by connecting.

### Photo sources / 照片数据源

Gallery supports read-only NAS folders and HTTPS WebDAV sources, random photo
preview, and device-bound scheduled random photos. Source definitions are persisted
in SQLite. See [setup and limits](docs/PHOTO_SOURCES.md).
NAS 本地目录需要显式只读挂载；网盘先支持通用 WebDAV，也可使用同步到 NAS 的目录。

### Build networking and cache

The Dockerfile retains the pinned Python base, locked Python dependencies,
`fonts-noto-cjk`, Debian repository signature checks, and offline artifact
SHA256 verification. Both production and CI use `scripts/build-verify.sh`,
which enables BuildKit and retains `docker compose build --pull`.

APT mirror hosts are configurable through `DEBIAN_MIRROR` and
`DEBIAN_SECURITY_MIRROR` (hosts only, without scheme or path). Defaults are
`deb.debian.org`; HTTPS is used for both Debian and security repositories.
For mainland-China NAS builds, use the shared profile:

```sh
. ./deploy/build-cn.env
./scripts/build-verify.sh
```

CI loads this same profile, using Tsinghua TUNA for both repositories unless
Gitea repository variables `DEBIAN_MIRROR` / `DEBIAN_SECURITY_MIRROR` override
it. Production may also set the two hosts directly in `.env` when calling the
build script without the profile. An alternate mainland mirror is
`mirrors.ustc.edu.cn`; availability and throughput depend on the Runner network.
No fallback disables signature verification or removes packages.

Layer order stays: pinned base → APT/font dependencies → `requirements.lock`
→ pip dependencies → application/vendor source → build metadata.
A source-only change therefore reuses the APT and pip layers, provided the
same builder, base digest, mirror arguments, and optional offline artifacts
are used. Changing the lockfile rebuilds pip and later layers, not APT.
Changing the mirror or APT instructions rebuilds APT and later layers.

Previously an APT layer miss redownloaded indexes/packages, and pip used
`--no-cache-dir`. Now locked BuildKit mounts retain APT indexes, downloaded
`.deb` files, and pip downloads/wheels outside the final image. Debian's
`docker-clean` hook is disabled so it cannot discard cached packages.
Re-executed APT steps still run `apt-get update` and validate repository
signatures; cached packages and pip artifacts can avoid repeat large downloads.
The initial build after this Dockerfile change will miss dependency layers;
subsequent builds benefit. No measured speedup is claimed before CI completes.

Caches belong to the Docker builder, not the ephemeral CI workspace. Keep the
NAS Runner's builder storage across jobs; removing the builder, pruning caches,
or using a fresh Docker daemon loses the benefit. Cache mounts are disposable:
an empty cache still builds correctly. `--pull` remains enabled; a pinned base
digest check alone does not invalidate dependency layers. To deliberately
refresh installed APT versions, use `docker compose build --pull --no-cache`.

### CI with a host Docker socket

Run `python scripts/ci-compose-verify.py` after loading the build mirror profile.
The script derives a temporary Compose document from production settings, with
an absolute client-side build context, a UUID project name, a unique CI image
tag, and project-scoped `data` / `config` named volumes. It removes production
bind mounts, the fixed container name, host ports, and the production `.env`
reference. A fresh test token, mock driver, and disabled device writes are
explicitly configured. Docker initializes empty named volumes from the image's
owned `/data` and `/config` directories; the service remains UID/GID 10333.

Build contexts are sent by the Docker client, so the job workspace can remain
inside the job container. Runtime storage is managed by the daemon; no matching
NAS host `/workspace` directory is needed. HTTP readiness and the unmodified
nine-endpoint smoke script run inside the service with `compose exec -T`.
Restart is followed by another readiness check and the same smoke test.

A `finally` block removes the test containers, network, volumes, and unique
image tag on success or failure. The workflow records the private temporary
Compose path in `GITHUB_ENV` and repeats cleanup in an `always()` step, covering
normal cancellation/interruption. Failed cleanup retains the manifest so the
fallback can retry. Abrupt host/daemon loss or a forcibly terminated entire job
can prevent both cleanup paths; no CI script can guarantee cleanup in that case.
No global prune or production resource removal is performed. Keep builder
caches: only the run-specific image tag is removed.

### CI Job pip indexes versus Docker build indexes

The Job bootstrap runs `python scripts/ci-install.py` before Ruff/mypy/tests.
It needs only the Python standard library, so it runs before PyYAML or tooling
is installed. `CI_PIP_INDEX_URL` and `CI_PIP_FALLBACK_INDEX_URL` are optional
Gitea repository variables. Defaults are the TUNA HTTPS PyPI simple endpoint
and official `https://pypi.org/simple`. Each is attempted separately; the
fallback repeats the unchanged editable `.[test]` installation, including
PEP 517 build isolation and `setuptools>=75`. No simultaneous extra index is
used, no build dependency is removed, and both failures fail CI.

Logs show inherited index/config/proxy hosts and the selected index host for
each attempt, with URL credentials and query strings removed. Bootstrap
disables inherited pip config files via `PIP_CONFIG_FILE=/dev/null`, explicitly
sets the selected index, and clears no-index/extra-index/find-links, insecure
trusted-host, and no-build-isolation environment settings. Constraints, TLS
certificate environment settings, and proxy settings remain in effect.

These `CI_PIP_*` settings apply only to the Job's pip install. They are not
passed into Docker build arguments or persisted into the production image.
The Dockerfile keeps its own cached/offline dependency installation and logs
the Docker pip index host for online installs. Debian mirror variables affect
APT only and are loaded later in the Docker verification step.

If dependency installation fails, the always() cleanup step still executes.
It skips the Python/PyYAML cleanup script when no CI Compose manifest exists,
as no Docker verification resources have been created at that point.

Production CD: [safe trial, configuration, and rollback](docs/deployment/GITEA_CD.md).