Files
pigway-frame-flow/docs/EXTENSIONS.md
T

89 lines
10 KiB
Markdown
Raw Permalink Normal View History

# Optional integrations / 可选连接
Frame Flow continues to run without AI, a cloud drive, calendar or Home Assistant. API-key frame control and official-account widget subscriptions remain separate integrations.
## Layout Studio / 排版工作室
Choose 1–12 local photos or import several JPG/PNG/WebP files at once. Imports preserve the original aspect ratio. Choose portrait (1200×1600) or landscape (1600×1200), grid/hero/poster, title and footer text. Generate a layout, drag regions or edit their coordinates and dimensions, then generate the final preview. Overlapping or out-of-bounds regions are rejected. Preview and download do not write to a frame; Display uses the existing queued Display Service.
可一次导入 1–12 张照片,选择横竖向及拼版样式,拖动图片或填写坐标调整位置。生成预览只生成本地图片;点击显示并确认后才上传画框。模板页的多图拼版和海报进入排版工作室。
AI layout is optional. Connections accepts a full OpenAI-compatible `https://…/v1/chat/completions` endpoint, model and API key. The endpoint must be public HTTPS. **Only layout text, asset IDs and dimensions are sent; photo pixels are not sent.** The returned JSON is validated against the same bounded layout schema. Failures are explicit; the user may select local layout. This is layout planning, not AI image generation. AI providers require separate user configuration and have not been tested against a paid account.
## Native cloud drives / 原生网盘
Photo Sources supports NAS directories, WebDAV, Google Drive, AliPan and Baidu Netdisk. Native drive connections require an already-authorized OAuth access token. Supply credentials through `FRAME_FLOW_SOURCE_*` (JSON) or the source credential form. They are stored only in `/config/source-<id>.secret` with mode 0600 and never returned in catalogs.
- Google Drive: location is the folder ID; JSON `{"access_token":"…"}`. Use a read-only Drive scope.
- AliPan: location is the parent file ID (`root` if appropriate); JSON `{"access_token":"…","drive_id":"…"}`. Your developer app must have the necessary approved read permissions.
- Baidu: location is an absolute cloud folder path; JSON `{"access_token":"…"}`. Permissions depend on the authorized developer app.
Queries are read-only, paginated, bounded to 200 pages and the configured catalog limit. Downloads are limited to selected images and approved provider download hosts. Browser OAuth authorization is not implemented yet. For automatic refresh, add `client_id`, `client_secret`, `refresh_token` and numeric Unix `expires_at` to file-based credentials. Near expiry, one serialized request refreshes the token at the fixed provider endpoint and atomically saves rotated credentials. Environment credentials are never rewritten: update the Secret when they expire. Failed refresh does not retry automatically. No account is connected merely by adding the connector. Google access also depends on network reachability from the NAS. Folder date captions, orientation metadata filtering and Synology People filtering currently require a local NAS source.
## Photo filtering / 照片筛选
Local Synology sources support shared or personal People space. Personal space requires a matching read-only photo-root mount and user permission; selecting the API space does not grant filesystem access. Optional thumbnail sharpness, brightness and perceptual similarity filters never change or delete source photos. These are heuristics, not closed-eye detection. All filters default off. Similarity history retains the last 200 selections for each source.
## Calendar and monitoring / 日历与监控
Connections configures Google Calendar, ICS with daily/weekly/monthly/yearly recurrence, Home Assistant state and HTTP JSON providers. Credentials are mode-0600 files. Public endpoints require an allowed HTTPS host; fixed LAN endpoints require explicit administrator opt-in. Results cache for five minutes. Create an initially disabled daily dashboard task, review its preview, then enable it. Google Calendar reads the next seven days. ICS expands the next seven days with RDATE, EXDATE, cancelled events, instance overrides and timezone handling. Frequencies faster than daily are rejected; calendars are capped at 2000 source events and 1000 expanded instances. Custom VTIMEZONE definitions and EXRULE are not implemented.
Home Assistant is read-only; NAS/Pi/server monitoring expects an external read-only JSON endpoint. No Docker socket or host monitoring privileges are added. These integrations do not require Ticktar.
## Official widgets / 官方小组件
Official-account authentication is required independently of the KoKonna API key. Only the authenticated catalog can be selected. Apply/cancel is an administrator action using the persistent operation queue, idempotency and the existing write guard. Cancellation does not automatically restore a previous photograph. Writes are sent once, followed by a read-only selection check. Timeout or failed confirmation becomes `UNKNOWN` and blocks subsequent writes until read-only reconciliation or explicit acknowledgement. No automated test uses a real device.
## Retention / 本地保留
Connections provides an opt-in generated-image/cache retention policy, disabled by default. Preview cleanup before enabling it. Manual imports, stars, active/unknown operation references, job/template references and known current/restore/override asset mappings are protected. This policy never deletes NAS originals or cloud images. The configured quota is a cleanup target, not a hard filesystem limit; protected files can exceed it. Unregistered cache files are not currently counted.
## Client scopes and Webhooks / 客户端权限与回调
The web UI continues using account/password sessions. Administrator-created external client tokens are shown once, stored as SHA-256 hashes and revocable. Scope middleware is shared with existing APIs.
Webhooks require explicitly allowed public HTTPS callback hosts. Each event has a stable ID; clients must deduplicate it. Signature header `X-Frame-Flow-Signature: sha256=<hex>` is HMAC-SHA256 over the exact UTF-8 request bytes. A signing key is shown once and stored in a mode-0600 file. Delivery state is durable, runs on a separate worker, and retries at most three times. Events before subscription creation are not replayed. Currently emitted events are `operation.completed`, `operation.failed` and `display.changed`; device online/offline and low-battery transitions are observed at most once per minute; cloud query failures do not imply offline. A received callback never initiates a frame write automatically.
## Other drivers / 其他画框
`devices.yml` may configure `driver: generic_http_frame`, `endpoint`, `token_env`, `allow_insecure_lan` and `enabled`. The adapter uses `/device`, `/images`, `/display` and `/images/{id}/content` with standardized Frame Flow device/image models. It retains the global write guard and never retries uncertain writes. Hardware interoperability needs a matching external implementation; no other vendor hardware is implied to be connected.
## Still pending / 尚未完成
Browser OAuth login; AI daily-art scheduling; custom VTIMEZONE/EXRULE; Home Assistant display driver; third-party live authorization and paid-model verification. Existing cloud/frame integrations remain operational independently of these optional capabilities.
Official references: [Google Drive list](https://developers.google.com/workspace/drive/api/reference/rest/v3/files/list), [Google Calendar events](https://developers.google.com/workspace/calendar/api/v3/reference/events/list), [Home Assistant REST](https://developers.home-assistant.io/docs/api/rest/), [AliPan developer docs](https://www.yuque.com/aliyundrive/zpfszx), [Baidu developer docs](https://pan.baidu.com/union/doc/nksg0sat9).
## AI image generation / AI 生图
Connections accepts a separate full `images/generations` HTTPS endpoint, model, supported size and secret key. `POST /api/v1/ai/images` uses a durable operation and Idempotency-Key. It requests exactly one image, accepts bounded base64 or an allowed HTTPS image URL, normalizes to JPEG and stores locally. No frame upload occurs automatically. This may incur provider charges. Failed or interrupted requests are never automatically re-sent. An interrupted generation is UNKNOWN but does not block frame writes. Compatibility depends on the chosen provider/model; no paid call has been made. See [official image API](https://developers.openai.com/api/reference/resources/images).
## MQTT
MQTT defaults disabled. Connections configures broker host/port/topic/TLS. TLS certificate verification remains enabled; plaintext requires explicit LAN opt-in. Secrets `FRAME_FLOW_MQTT_PASSWORD` and `FRAME_FLOW_MQTT_TOKEN` belong in deployment Secrets. The latter is a Frame Flow client token with `display:write`, not a KoKonna key. Ingress uses the same Display Service, write guard and durable idempotency as REST.
Publish a non-retained JSON message to the configured command topic:
```json
{"request_id":"unique-stable-command-id","content":{"type":"template","template":"quote","data":{"quote":"Hello"}}}
```
Replies appear on `<topic>/result` with operation_id or the standard error object. Poll the REST Operation API for completion. QoS duplicate messages with the same request_id cannot repeat the operation; retained commands are rejected. Automatic broker reconnection never re-sends frame writes. No broker is started or required for ordinary NAS operation.
### Docker dependency downloads
Docker builds use `DOCKER_PIP_INDEX_URL` (default: Tsinghua PyPI mirror) and
`DOCKER_PIP_FALLBACK_INDEX_URL` (default: official PyPI). Configure these as
Gitea Variables for both Verify and production deployment. These public mirror
URLs must not contain credentials; build arguments are visible in build metadata.
CI tooling separately uses `CI_PIP_INDEX_URL` and `CI_PIP_FALLBACK_INDEX_URL`.
A failed mirror installation retries the same locked requirements against the
fallback, without extra indexes, version downgrades, or disabling build isolation.
Logs show index hosts and redact URL credentials/query parameters.
APT/font layers remain cached when Python dependencies change. The Python layer
changes only when the lock, installer, base image, or mirror arguments change;
application source changes reuse it. BuildKit pip cache survives layer misses.
An explicitly supplied, checksum-verified offline wheel directory still takes
precedence and never falls back to the network.