Add optional AI generation, MQTT, OAuth refresh and calendar recurrence

This commit is contained in:
Pigway
2026-10-08 23:44:07 +08:00
parent 69a0841868
commit 8209aadbb8
24 changed files with 1113 additions and 37 deletions
+1 -1
View File
@@ -178,7 +178,7 @@ status is recorded honestly in docs/ACCEPTANCE.md.
## Roadmap
Native OAuth login/refresh, full ICS recurrence, MQTT, AI image generation/daily art, additional hardware adapters and device transition producers remain pending. See [optional capabilities and exact limits](docs/EXTENSIONS.md).
Browser OAuth login, AI daily-art scheduling, custom ICS timezones/EXRULE and additional hardware-specific adapters remain pending. MQTT, bounded recurrence, token refresh, transition events and single-image AI generation are optional capabilities. See [optional capabilities and exact limits](docs/EXTENSIONS.md).
## Layout Studio and optional integrations
+20 -4
View File
@@ -18,7 +18,7 @@ Photo Sources supports NAS directories, WebDAV, Google Drive, AliPan and Baidu N
- AliPan: location is the parent file ID (`root` if appropriate); JSON `{"access_token":"…","drive_id":"…"}`. Your developer app must have the necessary approved read permissions.
- Baidu: location is an absolute cloud folder path; JSON `{"access_token":"…"}`. Permissions depend on the authorized developer app.
Queries are read-only, paginated, bounded to 200 pages and the configured catalog limit. Downloads are limited to selected images and approved provider download hosts. Tokens expire: browser OAuth authorization and refresh-token rotation are **not implemented yet**. Expired tokens produce an explicit connection error. No account is connected merely by adding the connector. Google access also depends on network reachability from the NAS. Folder date captions, orientation metadata filtering and Synology People filtering currently require a local NAS source.
Queries are read-only, paginated, bounded to 200 pages and the configured catalog limit. Downloads are limited to selected images and approved provider download hosts. Browser OAuth authorization is not implemented yet. For automatic refresh, add `client_id`, `client_secret`, `refresh_token` and numeric Unix `expires_at` to file-based credentials. Near expiry, one serialized request refreshes the token at the fixed provider endpoint and atomically saves rotated credentials. Environment credentials are never rewritten: update the Secret when they expire. Failed refresh does not retry automatically. No account is connected merely by adding the connector. Google access also depends on network reachability from the NAS. Folder date captions, orientation metadata filtering and Synology People filtering currently require a local NAS source.
## Photo filtering / 照片筛选
@@ -26,7 +26,7 @@ Local Synology sources support shared or personal People space. Personal space r
## Calendar and monitoring / 日历与监控
Connections configures Google Calendar, basic non-recurring ICS, Home Assistant state and HTTP JSON providers. Credentials are mode-0600 files. Public endpoints require an allowed HTTPS host; fixed LAN endpoints require explicit administrator opt-in. Results cache for five minutes. Create an initially disabled daily dashboard task, review its preview, then enable it. Google Calendar reads the next seven days. Current ICS recurring events return `UNSUPPORTED_CAPABILITY`; they are not silently omitted.
Connections configures Google Calendar, ICS with daily/weekly/monthly/yearly recurrence, Home Assistant state and HTTP JSON providers. Credentials are mode-0600 files. Public endpoints require an allowed HTTPS host; fixed LAN endpoints require explicit administrator opt-in. Results cache for five minutes. Create an initially disabled daily dashboard task, review its preview, then enable it. Google Calendar reads the next seven days. ICS expands the next seven days with RDATE, EXDATE, cancelled events, instance overrides and timezone handling. Frequencies faster than daily are rejected; calendars are capped at 2000 source events and 1000 expanded instances. Custom VTIMEZONE definitions and EXRULE are not implemented.
Home Assistant is read-only; NAS/Pi/server monitoring expects an external read-only JSON endpoint. No Docker socket or host monitoring privileges are added. These integrations do not require Ticktar.
@@ -42,7 +42,7 @@ Connections provides an opt-in generated-image/cache retention policy, disabled
The web UI continues using account/password sessions. Administrator-created external client tokens are shown once, stored as SHA-256 hashes and revocable. Scope middleware is shared with existing APIs.
Webhooks require explicitly allowed public HTTPS callback hosts. Each event has a stable ID; clients must deduplicate it. Signature header `X-Frame-Flow-Signature: sha256=<hex>` is HMAC-SHA256 over the exact UTF-8 request bytes. A signing key is shown once and stored in a mode-0600 file. Delivery state is durable, runs on a separate worker, and retries at most three times. Events before subscription creation are not replayed. Currently emitted events are `operation.completed`, `operation.failed` and `display.changed`; device transition producers remain pending. A received callback never initiates a frame write automatically.
Webhooks require explicitly allowed public HTTPS callback hosts. Each event has a stable ID; clients must deduplicate it. Signature header `X-Frame-Flow-Signature: sha256=<hex>` is HMAC-SHA256 over the exact UTF-8 request bytes. A signing key is shown once and stored in a mode-0600 file. Delivery state is durable, runs on a separate worker, and retries at most three times. Events before subscription creation are not replayed. Currently emitted events are `operation.completed`, `operation.failed` and `display.changed`; device online/offline and low-battery transitions are observed at most once per minute; cloud query failures do not imply offline. A received callback never initiates a frame write automatically.
## Other drivers / 其他画框
@@ -50,6 +50,22 @@ Webhooks require explicitly allowed public HTTPS callback hosts. Each event has
## Still pending / 尚未完成
Native OAuth login/refresh; AI image generation/daily art; MQTT transport; full ICS recurrence; Home Assistant display driver; device-transition event polling; browser visual acceptance and third-party live authorization. Existing cloud/frame integrations remain operational independently of these optional capabilities.
Browser OAuth login; AI daily-art scheduling; custom VTIMEZONE/EXRULE; Home Assistant display driver; third-party live authorization and paid-model verification. Existing cloud/frame integrations remain operational independently of these optional capabilities.
Official references: [Google Drive list](https://developers.google.com/workspace/drive/api/reference/rest/v3/files/list), [Google Calendar events](https://developers.google.com/workspace/calendar/api/v3/reference/events/list), [Home Assistant REST](https://developers.home-assistant.io/docs/api/rest/), [AliPan developer docs](https://www.yuque.com/aliyundrive/zpfszx), [Baidu developer docs](https://pan.baidu.com/union/doc/nksg0sat9).
## AI image generation / AI 生图
Connections accepts a separate full `images/generations` HTTPS endpoint, model, supported size and secret key. `POST /api/v1/ai/images` uses a durable operation and Idempotency-Key. It requests exactly one image, accepts bounded base64 or an allowed HTTPS image URL, normalizes to JPEG and stores locally. No frame upload occurs automatically. This may incur provider charges. Failed or interrupted requests are never automatically re-sent. An interrupted generation is UNKNOWN but does not block frame writes. Compatibility depends on the chosen provider/model; no paid call has been made. See [official image API](https://developers.openai.com/api/reference/resources/images).
## MQTT
MQTT defaults disabled. Connections configures broker host/port/topic/TLS. TLS certificate verification remains enabled; plaintext requires explicit LAN opt-in. Secrets `FRAME_FLOW_MQTT_PASSWORD` and `FRAME_FLOW_MQTT_TOKEN` belong in deployment Secrets. The latter is a Frame Flow client token with `display:write`, not a KoKonna key. Ingress uses the same Display Service, write guard and durable idempotency as REST.
Publish a non-retained JSON message to the configured command topic:
```json
{"request_id":"unique-stable-command-id","content":{"type":"template","template":"quote","data":{"quote":"Hello"}}}
```
Replies appear on `<topic>/result` with operation_id or the standard error object. Poll the REST Operation API for completion. QoS duplicate messages with the same request_id cannot repeat the operation; retained commands are rejected. Automatic broker reconnection never re-sends frame writes. No broker is started or required for ordinary NAS operation.
+4
View File
@@ -110,7 +110,9 @@ def create_app(settings=None, start_worker=True):
async def lifespan(app):
if start_worker:
scheduler.start()
app.state.mqtt.start()
yield
app.state.mqtt.stop()
scheduler.stop()
db.close()
@@ -128,6 +130,7 @@ def create_app(settings=None, start_worker=True):
install_extensions(app, service)
install_accounts(app, db)
app.state.service = service
app.state.worker_enabled = start_worker
app.state.scheduler = scheduler
app.add_middleware(
CORSMiddleware,
@@ -1262,6 +1265,7 @@ def create_app(settings=None, start_worker=True):
if filename not in [
"app.js",
"features.js",
"extensions.js",
"style.css",
"i18n.js",
"swagger-ui-bundle.js",
+9 -7
View File
@@ -104,9 +104,9 @@ class DisplayService:
)
return existing["operation_id"]
if (
action not in ("render", "local-upload")
action not in ("render", "render-ai", "local-upload")
and tx.execute(
"SELECT id FROM operations WHERE device_id=? AND status='UNKNOWN' AND write_started=1 LIMIT 1",
"SELECT id FROM operations WHERE device_id=? AND status='UNKNOWN' AND write_started=1 AND action!='render-ai' LIMIT 1",
(device_id,),
).fetchone()
):
@@ -178,7 +178,7 @@ class DisplayService:
def blocked(self, device_id):
return (
self.db.one(
"SELECT id FROM operations WHERE device_id=? AND status='UNKNOWN' AND write_started=1 LIMIT 1",
"SELECT id FROM operations WHERE device_id=? AND status='UNKNOWN' AND write_started=1 AND action!='render-ai' LIMIT 1",
(device_id,),
)
is not None
@@ -406,9 +406,11 @@ class DisplayService:
def process_one(self):
with self.lock:
for op in self.queued_operations():
if op["action"] not in ("render", "local-upload") and self.blocked(
op["device_id"]
):
if op["action"] not in (
"render",
"render-ai",
"local-upload",
) and self.blocked(op["device_id"]):
continue
payload = json.loads(op["payload"])
if op["action"] == "display":
@@ -460,7 +462,7 @@ class DisplayService:
driver = self.registry.get(device_id)
if action in self.operation_handlers:
return self.operation_handlers[action](op, payload)
if action in ("render", "local-upload"):
if action in ("render", "render-ai", "local-upload"):
self.update(id, "RENDERING")
from pydantic import TypeAdapter
+176
View File
@@ -0,0 +1,176 @@
"""Optional one-image generation, using the durable operation queue."""
import base64
import json
from io import BytesIO
from typing import Literal
from urllib.parse import urlsplit
from fastapi import Depends, Header, Request
from PIL import Image, ImageOps
from pydantic import Field
from pigway_frame_flow.core.auth import require
from pigway_frame_flow.errors import ErrorCode as E
from pigway_frame_flow.errors import FlowError
from pigway_frame_flow.extensions.api import AISecret
from pigway_frame_flow.extensions.secrets import save_secret
from pigway_frame_flow.models import Accepted, StrictModel
from pigway_frame_flow.renderers.images import MAX_IMAGE_PIXELS
from pigway_frame_flow.storage.database import encode
class ImageGenerationConfig(StrictModel):
endpoint: str = Field(max_length=2048)
model: str = Field(min_length=1, max_length=120)
size: Literal["1024x1024", "1024x1536", "1536x1024"] = "1024x1536"
class ImageGenerationInput(StrictModel):
prompt: str = Field(min_length=1, max_length=4000)
def install_ai_images(app, service):
def config():
row = service.db.one("SELECT payload FROM app_config WHERE key='ai_images'")
if not row:
raise FlowError(E.CONFIG_REQUIRED, "Configure image generation first", 409)
return ImageGenerationConfig.model_validate_json(row["payload"])
@app.get(
"/api/v1/settings/ai-images",
dependencies=[Depends(require("admin"))],
tags=["AI"],
)
def get_config():
row = service.db.one("SELECT payload FROM app_config WHERE key='ai_images'")
return {
"config": json.loads(row["payload"]) if row else None,
"key_configured": (
service.settings.config_dir / "ai-images.secret"
).exists(),
}
@app.put(
"/api/v1/settings/ai-images",
dependencies=[Depends(require("admin"))],
tags=["AI"],
)
def set_config(payload: ImageGenerationConfig):
url = urlsplit(payload.endpoint)
if (
url.scheme != "https"
or not url.hostname
or url.username
or url.password
or url.query
or url.fragment
or url.port not in (None, 443)
):
raise FlowError(
E.INVALID_REQUEST,
"Image generation requires a plain public HTTPS endpoint",
)
if url.hostname not in service.settings.image_url_hosts:
service.settings.image_url_hosts.append(url.hostname)
service.db.execute(
"INSERT OR REPLACE INTO app_config VALUES('image_url_hosts',?)",
(encode(service.settings.image_url_hosts),),
)
service.db.execute(
"INSERT OR REPLACE INTO app_config VALUES('ai_images',?)",
(payload.model_dump_json(),),
)
return payload
@app.put(
"/api/v1/settings/ai-images/key",
status_code=204,
dependencies=[Depends(require("admin"))],
tags=["AI"],
)
def set_key(payload: AISecret):
key = payload.key.get_secret_value()
save_secret(service.settings.config_dir / "ai-images.secret", key)
service.settings.additional_secrets.append(key)
@app.post(
"/api/v1/ai/images",
status_code=202,
response_model=Accepted,
dependencies=[Depends(require("render:write"))],
tags=["AI"],
)
def generate(
payload: ImageGenerationInput,
request: Request,
idempotency_key: str | None = Header(default=None),
):
config()
if not (service.settings.config_dir / "ai-images.secret").exists():
raise FlowError(
E.CONFIG_REQUIRED, "Configure image generation key first", 409
)
id = service.submit(
"render-ai",
payload.model_dump(),
service.settings.device_id,
request.state.client_id,
request.state.request_id,
idempotency_key,
)
return Accepted(operation_id=id, job_id=id, status="accepted")
def execute(op, payload):
service.update(op["id"], "RENDERING")
conf = config()
key = (service.settings.config_dir / "ai-images.secret").read_text().strip()
service.settings.additional_secrets.append(key)
service.db.execute(
"UPDATE operations SET write_started=1 WHERE id=?", (op["id"],)
)
try:
raw = service.assets.fetch_url(
conf.endpoint,
method="POST",
headers={
"Authorization": "Bearer " + key,
"Content-Type": "application/json",
},
body=json.dumps(
{
"model": conf.model,
"prompt": payload["prompt"],
"size": conf.size,
"n": 1,
}
).encode(),
limit=service.settings.max_bytes,
)
result = json.loads(raw)["data"]
if len(result) != 1:
raise ValueError()
if result[0].get("b64_json"):
image_data = base64.b64decode(result[0]["b64_json"], validate=True)
else:
image_data = service.assets.fetch_url(result[0]["url"])
with Image.open(BytesIO(image_data)) as image:
if image.width * image.height > MAX_IMAGE_PIXELS:
raise ValueError()
rgba = ImageOps.exif_transpose(image).convert("RGBA")
rgb = Image.new("RGB", rgba.size, "white")
rgb.paste(rgba, mask=rgba.getchannel("A"))
output = BytesIO()
rgb.save(output, "JPEG", quality=93)
asset = service.assets.put(output.getvalue(), "ai-art.jpg", "generated")
except Exception:
raise FlowError(
E.RENDER_FAILED,
"Image generation failed; request may have been billed. No automatic retry was sent.",
502,
) from None
service.update(op["id"], "COMPLETED", asset)
service.db.history(op, "COMPLETED")
service.db.event("operation.completed", {"operation_id": op["id"]})
service.operation_handlers["render-ai"] = execute
+9
View File
@@ -42,6 +42,15 @@ def install_extensions(app, service):
from pigway_frame_flow.extensions.providers import install_providers
install_providers(app, service)
from pigway_frame_flow.extensions.ai_images import install_ai_images
install_ai_images(app, service)
from pigway_frame_flow.extensions.mqtt import install_mqtt
install_mqtt(app, service)
from pigway_frame_flow.extensions.device_events import DeviceObserver
service.device_observer = DeviceObserver(service)
db, settings = service.db, service.settings
studio = LayoutStudio(service.renderer)
service.local_retention = LocalRetention(service)
@@ -0,0 +1,49 @@
"""Low-frequency read-only observation emits transitions, never frame writes."""
import json
from pigway_frame_flow.storage.database import encode
class DeviceObserver:
def __init__(self, service):
self.service = service
def tick(self):
for id, driver in self.service.registry.drivers.items():
try:
device = driver.device()
except Exception:
# Cloud query failure is not evidence that the physical device is offline.
continue
low = (
device.power.battery_percent is not None
and device.power.battery_percent <= 20
)
state = {"status": device.status, "low_battery": low}
key = "device-observation:" + id
row = self.service.db.one(
"SELECT payload FROM app_config WHERE key=?", (key,)
)
previous = json.loads(row["payload"]) if row else None
if previous == state:
continue
if previous:
if previous["status"] != state["status"] and state["status"] in (
"online",
"offline",
):
self.service.db.event(
"device." + state["status"], {"device_id": id}
)
if low and not previous["low_battery"]:
self.service.db.event(
"device.low_battery",
{
"device_id": id,
"battery_percent": device.power.battery_percent,
},
)
self.service.db.execute(
"INSERT OR REPLACE INTO app_config VALUES(?,?)", (key, encode(state))
)
+176
View File
@@ -0,0 +1,176 @@
"""Opt-in MQTT ingress shares the display queue, scopes and idempotency."""
import hashlib
import json
from fastapi import Depends
from pydantic import Field
from pigway_frame_flow.core.auth import require
from pigway_frame_flow.errors import ErrorCode as E
from pigway_frame_flow.errors import FlowError
from pigway_frame_flow.models import DisplayRequest, StrictModel
from pigway_frame_flow.settings import secret
from pigway_frame_flow.storage.database import encode
class MQTTConfig(StrictModel):
enabled: bool = False
host: str = Field(default="", pattern=r"^[a-zA-Z0-9.:-]*$", max_length=253)
port: int = Field(default=8883, ge=1, le=65535)
tls: bool = True
allow_insecure_lan: bool = False
topic: str = Field(default="frame-flow/display", pattern=r"^[a-zA-Z0-9_/-]{1,120}$")
username: str = Field(default="", max_length=120)
class MQTTBridge:
def __init__(self, service):
self.service = service
self.client = None
self.connected = False
def config(self):
row = self.service.db.one("SELECT payload FROM app_config WHERE key='mqtt'")
return MQTTConfig.model_validate_json(row["payload"]) if row else MQTTConfig()
def accept(self, payload, retained=False):
if retained:
raise FlowError(E.INVALID_REQUEST, "Retained display commands are rejected")
if len(payload) > 65536:
raise FlowError(E.INVALID_REQUEST, "MQTT command exceeds size limit")
token = secret("FRAME_FLOW_MQTT_TOKEN")
row = (
self.service.db.one(
"SELECT * FROM clients WHERE token_hash=? AND enabled=1",
(hashlib.sha256(token.encode()).hexdigest(),),
)
if token
else None
)
if not row:
raise FlowError(
E.UNAUTHORIZED, "Configure an enabled Frame Flow MQTT client token", 401
)
scopes = json.loads(row["permissions"])
if "admin" not in scopes and "display:write" not in scopes:
raise FlowError(E.FORBIDDEN, "MQTT client lacks display:write", 403)
self.service.settings.additional_secrets.append(token)
data = json.loads(payload)
request_id = data.pop("request_id", "")
if not isinstance(request_id, str) or not request_id or len(request_id) > 120:
raise FlowError(
E.INVALID_REQUEST, "MQTT commands require a stable request_id"
)
request = DisplayRequest.model_validate(data)
self.service.validate(request)
id = self.service.submit(
"display",
request.model_dump(mode="json"),
request.device_id,
row["client_id"],
self.service.settings.sanitize(request_id),
"mqtt-" + hashlib.sha256(request_id.encode()).hexdigest(),
)
return {"operation_id": id, "status": "accepted"}
def start(self):
config = self.config()
if not config.enabled:
return
if not config.host or (not config.tls and not config.allow_insecure_lan):
raise FlowError(
E.CONFIG_REQUIRED,
"MQTT requires host and TLS or explicit LAN opt-in",
409,
)
import paho.mqtt.client as mqtt
self.client = mqtt.Client(
mqtt.CallbackAPIVersion.VERSION2,
client_id="frame-flow-"
+ hashlib.sha256(str(self.service.settings.data_dir).encode()).hexdigest()[
:12
],
)
password = secret("FRAME_FLOW_MQTT_PASSWORD")
self.service.settings.additional_secrets.append(password)
if config.username:
self.client.username_pw_set(config.username, password)
if config.tls:
self.client.tls_set()
self.client.reconnect_delay_set(min_delay=5, max_delay=120)
def connected(client, userdata, flags, reason_code, properties):
self.connected = not reason_code.is_failure
if self.connected:
client.subscribe(config.topic, qos=1)
def disconnected(client, userdata, flags, reason_code, properties):
self.connected = False
def message(client, userdata, message):
try:
response = self.accept(message.payload, message.retain)
except FlowError as exc:
response = {"error": {"code": exc.code, "message": exc.message}}
except Exception:
response = {
"error": {
"code": E.INVALID_REQUEST,
"message": "Invalid MQTT display request",
}
}
client.publish(
config.topic + "/result", encode(response), qos=1, retain=False
)
self.client.on_connect = connected
self.client.on_disconnect = disconnected
self.client.on_message = message
self.client.connect_async(config.host, config.port, keepalive=60)
self.client.loop_start()
def stop(self):
if self.client:
self.client.disconnect()
self.client.loop_stop()
self.client = None
self.connected = False
def install_mqtt(app, service):
service.mqtt = MQTTBridge(service)
app.state.mqtt = service.mqtt
@app.get(
"/api/v1/integrations/mqtt",
dependencies=[Depends(require("admin"))],
tags=["MQTT"],
)
def state():
return {
"config": service.mqtt.config().model_dump(),
"connected": service.mqtt.connected,
}
@app.put(
"/api/v1/integrations/mqtt",
dependencies=[Depends(require("admin"))],
tags=["MQTT"],
)
def configure(payload: MQTTConfig):
if payload.enabled and (
not payload.host or not payload.tls and not payload.allow_insecure_lan
):
raise FlowError(
E.INVALID_REQUEST, "MQTT needs a host and TLS or explicit LAN opt-in"
)
service.mqtt.stop()
service.db.execute(
"INSERT OR REPLACE INTO app_config VALUES('mqtt',?)",
(payload.model_dump_json(),),
)
if getattr(app.state, "worker_enabled", False):
service.mqtt.start()
return state()
@@ -2,6 +2,7 @@
import json
import re
import time
from pathlib import Path
from urllib.parse import urlencode, urlsplit
@@ -30,6 +31,29 @@ class CloudDrives:
for key in ["access_token", "refresh_token", "client_secret"]:
if value.get(key):
self.settings.additional_secrets.append(value[key])
if (
value.get("expires_at")
and float(value["expires_at"]) <= time.time() + 60
):
if source.credential_env:
raise FlowError(
E.CONFIG_REQUIRED,
"Environment OAuth token expired; update its secret",
409,
)
from pigway_frame_flow.providers.oauth import OAuthTokens
if not hasattr(self.photos, "oauth_tokens"):
self.photos.oauth_tokens = OAuthTokens(
self.settings, self.photos.assets
)
with self.photos.oauth_tokens.lock:
latest = json.loads(path.read_text())
value = (
latest
if float(latest.get("expires_at", 0)) > time.time() + 60
else self.photos.oauth_tokens.refresh(source.kind, latest, path)
)
return value
except (ValueError, TypeError, AttributeError):
raise FlowError(
+3 -20
View File
@@ -139,26 +139,9 @@ class ExternalProviders:
)
result: dict
if config.kind == "ics":
events = []
text = (
raw.decode("utf-8-sig").replace("\r\n ", "").replace("\r\n", "\n")
)
for block in text.split("BEGIN:VEVENT")[1:]:
block = block.split("END:VEVENT")[0]
values = {
line.split(":", 1)[0].split(";")[0]: line.split(":", 1)[1]
for line in block.splitlines()
if ":" in line
}
if "RRULE" in values:
raise FlowError(
E.UNSUPPORTED_CAPABILITY,
"Recurring ICS events require a calendar provider with expanded instances",
422,
)
if values.get("SUMMARY"):
events.append(values["SUMMARY"].replace("\\n", " "))
result = {"events": events[:100]}
from pigway_frame_flow.providers.ical import calendar_events
result = calendar_events(raw)
elif config.kind == "google-calendar":
result = {
"events": [
+101
View File
@@ -0,0 +1,101 @@
"""Bounded read-only iCalendar expansion for the next seven days."""
import re
from datetime import UTC, datetime, timedelta
from zoneinfo import ZoneInfo
from dateutil.rrule import rrulestr
from pigway_frame_flow.errors import ErrorCode as E
from pigway_frame_flow.errors import FlowError
def calendar_events(raw, now=None):
now = now or datetime.now(UTC)
end = now + timedelta(days=7)
text = re.sub(r"\r?\n[ \t]", "", raw.decode("utf-8-sig"))
blocks = text.split("BEGIN:VEVENT")[1:]
if len(blocks) > 2000:
raise FlowError(E.INVALID_REQUEST, "Calendar exceeds 2000 events")
def stamp(key, value):
zone = re.search(r"TZID=([^;:]+)", key)
tz = ZoneInfo(zone[1]) if zone else ZoneInfo("Asia/Shanghai")
if len(value) == 8:
return datetime.strptime(value, "%Y%m%d").replace(tzinfo=tz)
fmt = "%Y%m%dT%H%M%SZ" if value.endswith("Z") else "%Y%m%dT%H%M%S"
return datetime.strptime(value, fmt).replace(
tzinfo=UTC if value.endswith("Z") else tz
)
parsed: list[tuple] = []
for block in blocks:
props: dict[str, list[tuple[str, str]]] = {}
for line in block.split("END:VEVENT")[0].splitlines():
if ":" in line:
key, value = line.split(":", 1)
props.setdefault(key.split(";")[0], []).append((key, value))
if "DTSTART" not in props or "SUMMARY" not in props:
continue
start = stamp(*props["DTSTART"][0])
uid = props.get("UID", [(None, str(len(parsed)))])[0][1]
override = (
stamp(*props["RECURRENCE-ID"][0]) if "RECURRENCE-ID" in props else None
)
parsed.append((props, start, uid, override))
overrides = {
(uid, original.astimezone(UTC)) for _, _, uid, original in parsed if original
}
result = []
for props, start, uid, override in parsed:
if props.get("STATUS", [(None, "")])[0][1] == "CANCELLED":
continue
occurrences = [start]
if "RRULE" in props and not override:
rule = props["RRULE"][0][1]
if not re.search(r"(?:^|;)FREQ=(DAILY|WEEKLY|MONTHLY|YEARLY)(?:;|$)", rule):
raise FlowError(
E.UNSUPPORTED_CAPABILITY,
"Calendar frequency must be daily or slower",
422,
)
if now.year - start.year > 150:
raise FlowError(
E.INVALID_REQUEST, "Calendar recurrence starts too far in the past"
)
occurrences = rrulestr(rule, dtstart=start).between(
now - timedelta(days=1), end, inc=True
)
for key, value in props.get("RDATE", []):
occurrences.extend(stamp(key, item) for item in value.split(","))
excluded = {
stamp(key, item).astimezone(UTC)
for key, value in props.get("EXDATE", [])
for item in value.split(",")
}
title = (
props["SUMMARY"][0][1]
.replace("\\n", " ")
.replace("\\,", ",")
.replace("\\;", ";")
.replace("\\\\", "\\")
)
all_day = len(props["DTSTART"][0][1]) == 8
for occurrence in sorted(set(occurrences)):
instant = occurrence.astimezone(UTC)
lower = (
now.astimezone(occurrence.tzinfo).replace(
hour=0, minute=0, second=0, microsecond=0
)
if all_day
else now
)
if instant in excluded or (not override and (uid, instant) in overrides):
continue
if lower <= instant < end:
result.append({"title": title, "start": occurrence.isoformat()})
if len(result) > 1000:
raise FlowError(
E.INVALID_REQUEST, "Calendar expansion exceeds 1000 instances"
)
return {"events": sorted(result, key=lambda x: x["start"])}
+90
View File
@@ -0,0 +1,90 @@
"""Fixed-provider OAuth refresh. No secret payloads in durable operation history."""
import json
import threading
import time
from urllib.parse import urlencode
from pigway_frame_flow.errors import ErrorCode as E
from pigway_frame_flow.errors import FlowError
from pigway_frame_flow.extensions.secrets import save_secret
class OAuthTokens:
ENDPOINTS = {
"google-drive": "https://oauth2.googleapis.com/token",
"alipan": "https://openapi.aliyundrive.com/oauth/access_token",
"baidu": "https://openapi.baidu.com/oauth/2.0/token",
}
def __init__(self, settings, assets):
self.settings, self.assets = settings, assets
self.lock = threading.RLock()
def refresh(self, kind, credentials, path=None):
with self.lock:
if not all(
credentials.get(key)
for key in ("refresh_token", "client_id", "client_secret")
):
raise FlowError(
E.CONFIG_REQUIRED,
"OAuth refresh requires client ID, client secret and refresh token",
409,
)
values = {
key: credentials[key]
for key in ("refresh_token", "client_id", "client_secret")
}
values["grant_type"] = "refresh_token"
for key in ("access_token", "refresh_token", "client_secret"):
if credentials.get(key):
self.settings.additional_secrets.append(credentials[key])
endpoint = self.ENDPOINTS[kind]
host = endpoint.split("/")[2]
if host not in self.settings.image_url_hosts:
self.settings.image_url_hosts.append(host)
as_json = kind == "alipan"
body = (
json.dumps(values).encode() if as_json else urlencode(values).encode()
)
try:
raw = self.assets.fetch_url(
endpoint,
method="POST",
headers={
"Content-Type": "application/json"
if as_json
else "application/x-www-form-urlencoded"
},
body=body,
limit=65536,
)
response = json.loads(raw)
if (
not isinstance(response.get("access_token"), str)
or not response["access_token"]
):
raise ValueError()
expiry = int(response["expires_in"])
if not 60 <= expiry <= 366 * 86400:
raise ValueError()
updated = {
**credentials,
"access_token": response["access_token"],
"expires_at": time.time() + expiry,
}
if response.get("refresh_token"):
updated["refresh_token"] = response["refresh_token"]
for key in ("access_token", "refresh_token"):
if updated.get(key):
self.settings.additional_secrets.append(updated[key])
if path:
save_secret(path, json.dumps(updated))
return updated
except Exception:
raise FlowError(
E.CLOUD_UNAVAILABLE,
"OAuth refresh failed; reconnect or refresh credentials manually",
502,
) from None
+1
View File
@@ -258,6 +258,7 @@ class Scheduler:
def run_webhooks(self):
while not self.stop_event.is_set():
try:
self.service.device_observer.tick()
self.service.webhooks.tick()
except Exception:
logging.getLogger("frame-flow").error(
+8 -2
View File
@@ -5,7 +5,7 @@ async function layoutStudioPage(){
$('#main').innerHTML=`<div class="notice">${tr('一次选择 1–12 张图片,生成多图拼版或海报。AI 只接收文字与尺寸,不接收照片。预览不会上传到画框。','Choose 1–12 photos for a collage or poster. AI receives text and dimensions, never photos. Preview does not upload to the frame.')}</div><div class="columns"><section class="card"><h3>${tr('选择照片','Choose photos')}</h3><input id="studio-files" type="file" multiple accept="image/jpeg,image/png,image/webp"><p class="hint">${tr('可以一次上传多张,保留原始比例。或勾选已有本地照片。','Upload several photos at once, preserving aspect ratio, or select local photos.')}</p><div id="studio-assets" class="studio-assets">${assets.map(a=>`<label class="check-row"><input type="checkbox" value="${a.asset_id}">${esc(a.name)}</label>`).join('')}</div><div class="field"><label>${tr('标题','Title')}</label><input id="studio-title" maxlength="100"></div><div class="field"><label>${tr('底部文字','Footer text')}</label><textarea id="studio-message" maxlength="500"></textarea></div><div class="form-grid"><div class="field"><label>${tr('布局','Layout')}</label><select id="studio-style"><option value="grid">${tr('均匀拼版','Grid')}</option><option value="hero">${tr('一张主图+多张附图','Hero with supporting photos')}</option><option value="poster">${tr('海报','Poster')}</option></select></div><div class="field"><label>${tr('方向','Orientation')}</label><select id="studio-orientation"><option value="portrait">${tr('竖向','Portrait')}</option><option value="landscape">${tr('横向','Landscape')}</option></select></div></div><div class="field"><label>${t('profile')}</label><select id="studio-profile"><option value="color">${t('color')}</option><option value="grayscale">${t('grayscale')}</option></select></div><label class="check-row"><input id="studio-ai" type="checkbox">${tr('使用 AI 排版(先在连接中心配置)','Use AI layout (configure in Connections first)')}</label><div class="field"><label>${tr('AI 排版要求','AI layout instructions')}</label><textarea id="studio-prompt" maxlength="1500" placeholder="${tr('例如:第一张作为主图,其余大小错落','Example: feature the first photo with smaller supporting photos')}"></textarea></div><div class="actions"><button id="studio-plan" class="primary">${tr('规划布局','Plan layout')}</button><button id="studio-render" disabled>${t('render')}</button></div><p class="hint">${tr('规划后可拖动照片区域或修改坐标尺寸。区域不能重叠,超出范围的布局会被拒绝。','After planning, drag photo regions or edit coordinates and sizes. Overlapping or out-of-bounds layouts are rejected.')}</p><div id="studio-region-editor"></div></section><section class="card"><h3>${t('preview')}</h3><div id="studio-layout"></div><div id="studio-preview" class="empty">${tr('先规划布局,再生成最终图片','Plan the layout, then render the final image')}</div><div class="actions"><button id="studio-display" disabled>${t('display')}</button><button id="studio-download" disabled>${t('download')}</button></div></section></div>`;
let rendered=null;
const invalidate=()=>{rendered=null;$('#studio-display').disabled=true;$('#studio-download').disabled=true;};
$('#studio-assets').onchange=()=>{selected=new Set([...$('#studio-assets input:checked')].map(x=>x.value));plan=null;$('#studio-render').disabled=true;invalidate();};
$('#studio-assets').onchange=()=>{selected=new Set([...document.querySelectorAll('#studio-assets input:checked')].map(x=>x.value));plan=null;$('#studio-render').disabled=true;invalidate();};
$('#studio-files').onchange=busy(async()=>{const files=[...$('#studio-files').files];if(!files.length)return;const body=new FormData();files.forEach(f=>body.append('files',f));const uploaded=await api('/api/v1/layout/images',{method:'POST',body});for(const a of uploaded){const label=document.createElement('label');label.className='check-row';label.innerHTML=`<input type="checkbox" checked value="${a.asset_id}">${esc(a.name)}`;$('#studio-assets').append(label);}$('#studio-assets').onchange();toast(t('done'));});
async function showPlan(){
const box=$('#studio-layout');box.innerHTML='';box.className='studio-canvas';box.style.aspectRatio=plan.width+'/'+plan.height;
@@ -24,7 +24,7 @@ async function layoutStudioPage(){
}
async function connectionsPage(){
$('#main').innerHTML=`<div class="notice">${tr('可选连接不会影响 NAS 照片和现有设备。账号密钥保存在受限配置文件中,外部客户端密钥仅显示一次。','Optional connections do not affect NAS photos or existing devices. Provider keys use restricted config files; client tokens are shown once.')}</div>`;
await aiSettingsPanel();await storageSettingsPanel();await clientsPanel();await webhooksPanel();await externalProvidersPanel();
await aiSettingsPanel();await aiImagePanel();await storageSettingsPanel();await clientsPanel();await webhooksPanel();await externalProvidersPanel();await mqttPanel();
}
async function aiSettingsPanel(){
const config=await api('/api/v1/settings/ai-layout');const card=document.createElement('section');card.className='card';card.innerHTML=`<h3>${tr('AI 排版 · OpenAI 兼容','AI layout · OpenAI compatible')}</h3><p class="hint">${tr('只发送文字、尺寸和排版要求,不发送照片。填写完整 chat/completions 地址;使用服务商的模型名称。','Only text, dimensions and instructions are sent; never photos. Enter the full chat/completions URL and provider model name.')}</p><form id="ai-settings-form"><div class="field"><label>Endpoint</label><input id="ai-endpoint" type="url" value="${esc(config.endpoint)}" placeholder="https://…/v1/chat/completions"></div><div class="field"><label>Model</label><input id="ai-model" value="${esc(config.model)}"></div><div class="field"><label>${tr('API Key(留空保留已有密钥)','API key (blank keeps existing)')}</label><input id="ai-key" type="password" autocomplete="off"><p class="hint">${t('keyConfigured')}: ${t(config.key_configured?'yes':'no')}</p></div><button type="submit">${t('save')}</button></form>`;$('#main').append(card);$('#ai-settings-form').onsubmit=busy(async e=>{e.preventDefault();await api('/api/v1/settings/ai-layout',{method:'PUT',body:{endpoint:$('#ai-endpoint').value,model:$('#ai-model').value}});const key=$('#ai-key').value;$('#ai-key').value='';if(key)await api('/api/v1/settings/ai-layout/key',{method:'PUT',body:{key}});toast(t('done'));});
@@ -50,3 +50,9 @@ function addPhotoQualityFields(source){
const box=document.createElement('section');box.innerHTML=`<h3>${tr('照片筛选','Photo filtering')}</h3><p class="hint">${tr('按小图清晰度、亮度和相似度筛选,不改动或删除 NAS 原照片;无法保证识别闭眼。建议先用低阈值。个人空间还需要将相应目录只读挂载到 /photos。','Filters thumbnail sharpness, brightness and similarity without modifying or deleting originals. Closed-eye detection is not guaranteed. Start with low thresholds. Personal space also requires mounting its folder read-only at /photos.')}</p><div class="field"><label>${tr('人物空间','People space')}</label><select id="source-people-space"><option value="shared">${tr('共享空间','Shared space')}</option><option value="personal">${tr('个人空间','Personal space')}</option></select></div><div class="form-grid"><div class="field"><label>${tr('最低清晰度(0 不筛选)','Minimum sharpness (0 disables)')}</label><input id="source-sharpness" type="number" min="0" max="10000" value="${source?.minimum_sharpness||0}"></div><div class="field"><label>${tr('最低亮度','Minimum brightness')}</label><input id="source-bright-min" type="number" min="0" max="255" value="${source?.minimum_brightness||0}"></div><div class="field"><label>${tr('最高亮度','Maximum brightness')}</label><input id="source-bright-max" type="number" min="0" max="255" value="${source?.maximum_brightness??255}"></div></div><label class="check-row"><input id="source-dedup" type="checkbox" ${source?.avoid_duplicates?'checked':''}>${tr('避开最近 200 张的相似照片','Avoid photos similar to the last 200 selections')}</label><div class="field"><label>${tr('网盘 OAuth 凭据 JSON(可选;仅原生网盘)','Drive OAuth credentials JSON (optional; native drives only)')}</label><input id="source-oauth" type="password" autocomplete="off" placeholder='{"access_token":"…","drive_id":"…"}'></textarea></div>`;$('#photo-source-form .modal-actions').before(box);$('#source-people-space').value=source?.people_space||'shared';
}
async function aiImagePanel(){
const info=await api('/api/v1/settings/ai-images');const conf=info.config||{};const card=document.createElement('section');card.className='card';card.innerHTML=`<h3>${tr('AI 生图','AI image generation')}</h3><p class="hint">${tr('独立配置 images/generations 接口。每次仅生成一张并保存本地,不会自动显示到画框。请求可能收费,失败不自动重试。','Configure a separate images/generations endpoint. One image is saved locally per request, never automatically displayed. Requests may be billed; failures are not retried.')}</p><form id="image-ai-config"><div class="field"><label>Endpoint</label><input id="image-ai-url" type="url" required value="${esc(conf.endpoint||'')}"></div><div class="field"><label>Model</label><input id="image-ai-model" required value="${esc(conf.model||'')}"></div><div class="field"><label>Size</label><select id="image-ai-size"><option>1024x1536</option><option>1536x1024</option><option>1024x1024</option></select></div><div class="field"><label>API Key</label><input id="image-ai-key" type="password" autocomplete="off"><p class="hint">${t('keyConfigured')}: ${t(info.key_configured?'yes':'no')}</p></div><button type="submit">${t('save')}</button></form><form id="image-ai-generate"><div class="field"><label>${tr('画面描述','Image prompt')}</label><textarea id="image-ai-prompt" required maxlength="4000"></textarea></div><button type="submit">${tr('生成一张(可能收费)','Generate one image (may incur charges)')}</button></form><div id="image-ai-preview"></div>`;$('#main').append(card);$('#image-ai-size').value=conf.size||'1024x1536';$('#image-ai-config').onsubmit=busy(async e=>{e.preventDefault();await api('/api/v1/settings/ai-images',{method:'PUT',body:{endpoint:$('#image-ai-url').value,model:$('#image-ai-model').value,size:$('#image-ai-size').value}});const key=$('#image-ai-key').value;$('#image-ai-key').value='';if(key)await api('/api/v1/settings/ai-images/key',{method:'PUT',body:{key}});toast(t('done'));});$('#image-ai-generate').onsubmit=busy(async e=>{e.preventDefault();const result=await api('/api/v1/ai/images',{method:'POST',body:{prompt:$('#image-ai-prompt').value},headers:{'Idempotency-Key':'ai-image-'+clientUUID()}});const op=await waitOperation(result.operation_id);$('#image-ai-preview').innerHTML='<img class="preview" id="image-ai-result" alt="Generated image">';await blobImage($('#image-ai-result'),op.result.url);toast(tr('已存入本地图库','Saved to local gallery'));});
}
async function mqttPanel(){
const status=await api('/api/v1/integrations/mqtt');const c=status.config;const card=document.createElement('section');card.className='card';card.innerHTML=`<h3>MQTT · ${t(status.connected?'connected':'disconnected')}</h3><p class="hint">${tr('默认关闭。须配置环境 Secret FRAME_FLOW_MQTT_TOKEN(有 display:write 权限的客户端 Token),以及可选 FRAME_FLOW_MQTT_PASSWORD。消息必须携带 request_id;retained 消息被拒绝。结果发送到命令主题的 /result。','Disabled by default. Configure FRAME_FLOW_MQTT_TOKEN with display:write scope and optional FRAME_FLOW_MQTT_PASSWORD as environment secrets. Commands require request_id; retained commands are rejected. Results are published to the command topic plus /result.')}</p><form id="mqtt-config"><label class="check-row"><input id="mqtt-enabled" type="checkbox" ${c.enabled?'checked':''}>${t('enabled')}</label><div class="field"><label>Host</label><input id="mqtt-host" value="${esc(c.host)}"></div><div class="field"><label>Port</label><input id="mqtt-port" type="number" min="1" max="65535" value="${c.port}"></div><div class="field"><label>Topic</label><input id="mqtt-topic" value="${esc(c.topic)}"></div><div class="field"><label>${t('username')}</label><input id="mqtt-username" value="${esc(c.username)}"></div><label class="check-row"><input id="mqtt-tls" type="checkbox" ${c.tls?'checked':''}>TLS</label><label class="check-row"><input id="mqtt-lan" type="checkbox" ${c.allow_insecure_lan?'checked':''}>${tr('明确允许 LAN 明文连接','Explicitly allow plaintext LAN connection')}</label><button type="submit">${t('save')}</button></form>`;$('#main').append(card);$('#mqtt-config').onsubmit=busy(async e=>{e.preventDefault();await api('/api/v1/integrations/mqtt',{method:'PUT',body:{enabled:$('#mqtt-enabled').checked,host:$('#mqtt-host').value,port:Number($('#mqtt-port').value),topic:$('#mqtt-topic').value,username:$('#mqtt-username').value,tls:$('#mqtt-tls').checked,allow_insecure_lan:$('#mqtt-lan').checked}});toast(t('done'));});
}
+1 -1
View File
@@ -132,7 +132,7 @@ async function photoSourcesPanel(){
}
async function photoSourceForm(source=null){
showModal(`<h2>${t(source?'edit':'addSource')}</h2><form id="photo-source-form"><div class="field"><label for="source-id">ID</label><input id="source-id" pattern="[a-z0-9-]{1,60}" required value="${esc(source?.id||'')}" ${source?'readonly':''}></div><div class="field"><label for="source-name">${t('name')}</label><input id="source-name" required maxlength="120" value="${esc(source?.name||'')}"></div><div class="field"><label for="source-kind">${t('sourceKind')}</label><select id="source-kind"><option value="local">${t('nasDirectory')}</option><option value="webdav">WebDAV</option><option value="google-drive">Google Drive</option><option value="alipan">阿里云盘 / AliPan</option><option value="baidu">百度网盘 / Baidu</option></select></div><div class="field"><label for="source-location">${t('sourceLocation')}</label><input id="source-location" required maxlength="2048" value="${esc(source?.location||'.')}"><p class="hint">${t('sourceLocationHint')}</p></div><div class="field"><label for="source-credential">${t('sourceCredential')}</label><input id="source-credential" pattern="FRAME_FLOW_SOURCE_[A-Z0-9_]+" value="${esc(source?.credential_env||'')}" placeholder="FRAME_FLOW_SOURCE_CLOUD"><p class="hint">${t('sourceCredentialHint')}</p></div><div class="field"><label for="source-orientation">${t('photoOrientation')}</label><select id="source-orientation">${source&&!source.orientation?`<option value="">${t('orientationLegacy')}</option>`:''}<option value="auto">${t('orientationAuto')}</option><option value="landscape">${t('orientationLandscape')}</option><option value="portrait">${t('orientationPortrait')}</option></select><p class="hint">${t('orientationHint')}</p></div><div class="field"><label class="check-row"><input id="source-organized" type="checkbox" ${source?.organized_only?'checked':''}>${t('organizedPhotos')}</label><label class="check-row"><input id="source-caption" type="checkbox" ${source?.folder_caption?'checked':''}>${t('folderCaption')}</label><label class="check-row"><input id="source-synology-previews" type="checkbox" ${source?.synology_previews?'checked':''}>${t('synologyPreviews')}</label><p class="hint">${t('folderCaptionHint')}</p></div><div class="field"><label class="check-row"><input id="source-people-filter" type="checkbox" ${source?.people_filter?'checked':''}>${t('peopleFilter')}</label><p class="hint">${t('peopleHint')}</p><button id="source-people-load" type="button">${t('choosePeople')}</button><div id="source-people-panel"><p class="hint">${t('peopleSelected')}${(source?.person_ids||[]).length}</p></div><label for="source-person-match">${t('personMatch')}</label><select id="source-person-match"><option value="any">${t('personAny')}</option><option value="all">${t('personAll')}</option></select></div><div class="field"><label class="check-row"><input id="source-recursive" type="checkbox" ${source?.recursive===false?'':'checked'}>${t('sourceRecursive')}</label><label class="check-row"><input id="source-enabled" type="checkbox" ${source?.enabled===false?'':'checked'}>${t('enabled')}</label></div><div class="modal-actions">${buttons('modal-cancel','cancel')}<button type="submit" class="primary">${t('save')}</button></div></form>`);
$('#source-orientation').value=source?(source.orientation||''):'auto';const selectedPeople=new Set(source?.person_ids||[]);$('#source-person-match').value=source?.person_match||'any';$('#source-people-load').onclick=busy(async()=>{await photosPeoplePicker(selectedPeople);});$('#source-kind').value=source?.kind||'local';const update=()=>{const dav=$('#source-kind').value!=='local';$('#source-orientation').disabled=dav;$('#source-credential').disabled=!dav;$('#source-recursive').disabled=$('#source-kind').value==='webdav';$('#source-synology-previews').disabled=dav;$('#source-people-filter').disabled=dav;$('#source-people-load').disabled=dav;};update();$('#source-kind').onchange=update;addPhotoQualityFields(source);$('#source-people-space').onchange=()=>{selectedPeople.clear();$('#source-people-panel').innerHTML='';};
$('#source-orientation').value=source?(source.orientation||''):'auto';const selectedPeople=new Set(source?.person_ids||[]);$('#source-person-match').value=source?.person_match||'any';$('#source-people-load').onclick=busy(async()=>{await photosPeoplePicker(selectedPeople);});$('#source-kind').value=source?.kind||'local';const update=()=>{const dav=$('#source-kind').value!=='local';$('#source-orientation').disabled=dav;$('#source-credential').disabled=!dav;$('#source-recursive').disabled=$('#source-kind').value==='webdav';$('#source-synology-previews').disabled=dav;$('#source-organized').disabled=dav;$('#source-caption').disabled=dav;$('#source-people-filter').disabled=dav;$('#source-people-load').disabled=dav;};update();$('#source-kind').onchange=update;addPhotoQualityFields(source);$('#source-people-space').onchange=()=>{selectedPeople.clear();$('#source-people-panel').innerHTML='';};
$('#photo-source-form').onsubmit=busy(async e=>{e.preventDefault();const oauth=$('#source-oauth').value;const sourceId=$('#source-id').value;const native=['google-drive','alipan','baidu'].includes($('#source-kind').value);if(oauth&&native)JSON.parse(oauth);const dav=$('#source-kind').value!=='local';await api('/api/v1/photo-sources/'+$('#source-id').value,{method:'PUT',body:{name:$('#source-name').value,kind:$('#source-kind').value,location:$('#source-location').value,recursive:$('#source-kind').value!=='webdav'&&$('#source-recursive').checked,synology_previews:!dav&&$('#source-synology-previews').checked,catalog_limit:$('#source-organized').checked?100000:(source?.catalog_limit||10000),organized_only:!dav&&$('#source-organized').checked,folder_caption:!dav&&$('#source-caption').checked,orientation:dav?null:($('#source-orientation').value||null),people_filter:!dav&&$('#source-people-filter').checked,person_ids:[...selectedPeople],person_match:$('#source-person-match').value,people_space:$('#source-people-space').value,minimum_sharpness:Number($('#source-sharpness').value),minimum_brightness:Number($('#source-bright-min').value),maximum_brightness:Number($('#source-bright-max').value),avoid_duplicates:$('#source-dedup').checked,enabled:$('#source-enabled').checked,credential_env:dav?($('#source-credential').value||null):null}});if(oauth&&native){$('#source-oauth').value='';await api('/api/v1/photo-sources/'+sourceId+'/credentials',{method:'PUT',body:{key:oauth}});}$('#modal').close();await navigate('photo-sources');});
}
async function browseSource(id,offset=0,refresh=false){
+1 -1
View File
@@ -2,7 +2,7 @@
name = "pigway-frame-flow"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["fastapi==0.142.2", "uvicorn==0.54.0", "pillow==12.3.0", "requests==2.34.2", "croniter==6.2.4", "python-multipart==0.0.32", "pyyaml==6.0.3"]
dependencies = ["paho-mqtt==2.1.0", "python-dateutil==2.9.0.post0", "fastapi==0.142.2", "uvicorn==0.54.0", "pillow==12.3.0", "requests==2.34.2", "croniter==6.2.4", "python-multipart==0.0.32", "pyyaml==6.0.3"]
[project.optional-dependencies]
test = ["pytest==9.1.1", "httpx2==2.13.1", "mypy==2.3.1", "pip-audit==2.10.1", "types-requests==2.33.0.20260906", "types-PyYAML==6.0.12.20260906", "types-croniter==6.2.4.20260711", "ruff==0.16.9"]
[tool.pytest.ini_options]
+2 -1
View File
@@ -9,9 +9,10 @@ fastapi==0.142.2
h11==0.16.0
idna==3.20
opentelemetry-api==1.45.0
paho-mqtt==2.1.0
pillow==12.3.0
pydantic==2.13.5
pydantic-core==2.46.5
pydantic==2.13.5
python-dateutil==2.9.0.post0
python-multipart==0.0.32
pyyaml==6.0.3
+28
View File
@@ -0,0 +1,28 @@
/* Isolated Mock-only browser regression. Never touches the user's browser or frame. */
const assert=require('assert');const fs=require('fs');const {chromium}=require('playwright');
(async()=>{
const base=process.env.FRAME_FLOW_URL;assert(base,'FRAME_FLOW_URL required');
const env=fs.readFileSync(process.env.FRAME_FLOW_ENV_FILE,'utf8');const token=env.split('\n').find(x=>x.startsWith('FRAME_FLOW_TOKEN=')).slice(17);
const config=await (await fetch(base+'/api/v1/settings',{headers:{Authorization:'Bearer '+token}})).json();assert(config.mock===true,'Mock required');
const browser=await chromium.launch({headless:true,executablePath:process.env.CHROME_EXECUTABLE||'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'});
try{
const page=await browser.newPage();const errors=[];page.on('pageerror',e=>errors.push(e.message));
await page.goto(base);await page.locator('#username').fill(process.env.FRAME_FLOW_TEST_USERNAME);await page.locator('#password').fill(process.env.FRAME_FLOW_TEST_PASSWORD);await page.locator('#login-form button').click();await page.locator('.hero').waitFor();
let checks=0;
for(const width of [1440,768,390])for(const lang of ['zh','en'])for(const theme of ['light','dark']){
await page.setViewportSize({width,height:900});await page.evaluate(({lang,theme})=>{state.lang=lang;document.documentElement.dataset.theme=theme;shell();},{lang,theme});
for(const view of ['studio','connections']){
await page.evaluate(view=>navigate(view),view);await page.locator(view==='studio'?'#studio-plan':'#mqtt-config').waitFor();
assert(await page.evaluate(()=>document.documentElement.scrollWidth<=window.innerWidth),view+' overflow '+width);checks++;
const gaps=await page.locator('.actions').evaluateAll(rows=>rows.flatMap(row=>{const nodes=[...row.querySelectorAll('button')];return nodes.slice(1).map((node,i)=>{const a=nodes[i].getBoundingClientRect(),b=node.getBoundingClientRect();return Math.abs(a.top-b.top)<4?b.left-a.right:b.top-a.bottom;});}));
assert(gaps.every(g=>g>=8),view+' buttons touch '+width);checks++;
}
}
await page.evaluate(()=>navigate('studio'));await page.locator('#studio-files').setInputFiles({name:'fixture.png',mimeType:'image/png',buffer:Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+a1OIAAAAASUVORK5CYII=','base64')});
await page.locator('#studio-assets input:checked').waitFor();await page.locator('#studio-plan').click();await page.locator('#studio-layout img').waitFor();await page.locator('#studio-render').click();await page.locator('#studio-result').waitFor({timeout:30000});assert(await page.locator('#studio-download').isEnabled());checks++;
await page.locator('#studio-title').fill('Changed');assert(await page.locator('#studio-display').isDisabled());checks++;
assert.deepStrictEqual(errors,[]);checks++;
const out=process.env.FRAME_FLOW_SCREENSHOT_DIR;if(out){fs.mkdirSync(out,{recursive:true});await page.screenshot({path:out+'/studio-mobile.png',fullPage:true});await page.evaluate(()=>navigate('connections'));await page.locator('#mqtt-config').waitFor();await page.screenshot({path:out+'/connections-mobile.png',fullPage:true});}
console.log('Integration UI:',checks,'passed; 12 viewport/language/theme combinations; console errors:',errors.length);
}finally{await browser.close();}
})().catch(e=>{console.error(e.message);process.exit(1);});
+92
View File
@@ -141,6 +141,98 @@
"503"
]
},
"/api/v1/settings/ai-images": {
"get": [
"200",
"400",
"401",
"403",
"404",
"409",
"413",
"422",
"429",
"500",
"502",
"503"
],
"put": [
"200",
"400",
"401",
"403",
"404",
"409",
"413",
"422",
"429",
"500",
"502",
"503"
]
},
"/api/v1/settings/ai-images/key": {
"put": [
"204",
"400",
"401",
"403",
"404",
"409",
"413",
"422",
"429",
"500",
"502",
"503"
]
},
"/api/v1/ai/images": {
"post": [
"202",
"400",
"401",
"403",
"404",
"409",
"413",
"422",
"429",
"500",
"502",
"503"
]
},
"/api/v1/integrations/mqtt": {
"get": [
"200",
"400",
"401",
"403",
"404",
"409",
"413",
"422",
"429",
"500",
"502",
"503"
],
"put": [
"200",
"400",
"401",
"403",
"404",
"409",
"413",
"422",
"429",
"500",
"502",
"503"
]
},
"/api/v1/layout/images": {
"post": [
"201",
+104
View File
@@ -0,0 +1,104 @@
import base64
import json
from io import BytesIO
from PIL import Image
def test_generate_requires_configuration(client):
assert (
client.post("/api/v1/ai/images", json={"prompt": "Landscape"}).status_code
== 409
)
def test_generation_queued_once_and_never_displays(client, service, monkeypatch):
assert (
client.put(
"/api/v1/settings/ai-images",
json={
"endpoint": "https://example.com/v1/images/generations",
"model": "test-model",
},
).status_code
== 200
)
client.put("/api/v1/settings/ai-images/key", json={"key": "private-ai-key"})
image = BytesIO()
Image.new("RGB", (100, 200), "blue").save(image, "PNG")
calls = []
def fetch(url, **kwargs):
calls.append(kwargs)
return json.dumps(
{"data": [{"b64_json": base64.b64encode(image.getvalue()).decode()}]}
).encode()
monkeypatch.setattr(service.assets, "fetch_url", fetch)
headers = {"Idempotency-Key": "art-once"}
first = client.post(
"/api/v1/ai/images", json={"prompt": "Landscape"}, headers=headers
)
assert first.status_code == 202
assert (
client.post(
"/api/v1/ai/images", json={"prompt": "Landscape"}, headers=headers
).json()
== first.json()
)
service.process_one()
operation = service.operation(first.json()["operation_id"])
assert operation["status"] == "COMPLETED"
assert len(calls) == 1
assert json.loads(calls[0]["body"])["n"] == 1
assert service.registry.get("living-room").client.uploads == 0
assert "private-ai-key" not in client.get("/api/v1/settings/ai-images").text
def test_generation_failure_never_retries(client, service, monkeypatch):
client.put(
"/api/v1/settings/ai-images",
json={"endpoint": "https://example.com/v1/images/generations", "model": "test"},
)
client.put("/api/v1/settings/ai-images/key", json={"key": "private-ai-key"})
calls = []
def fail(*args, **kwargs):
calls.append(1)
raise TimeoutError("private-ai-key")
monkeypatch.setattr(service.assets, "fetch_url", fail)
result = client.post("/api/v1/ai/images", json={"prompt": "Landscape"})
service.process_one()
service.process_one()
op = service.operation(result.json()["operation_id"])
assert op["status"] == "FAILED"
assert len(calls) == 1
assert "private-ai-key" not in json.dumps(op)
def test_interrupted_generation_is_never_requeued(app, service, settings):
from pigway_frame_flow.api.app import create_app
id = service.submit(
"render-ai",
{"prompt": "One image"},
"living-room",
"admin",
"interrupted",
"one",
)
service.db.execute(
"UPDATE operations SET status='RENDERING',write_started=1 WHERE id=?", (id,)
)
restarted = create_app(settings, start_worker=False)
assert restarted.state.service.operation(id)["status"] == "UNKNOWN"
assert not restarted.state.service.blocked("living-room")
assert restarted.state.service.process_one() is False
restarted.state.service.db.close()
def test_extension_script_served(client):
response = client.get("/ui/extensions.js")
assert response.status_code == 200
assert "layoutStudioPage" in response.text
+31
View File
@@ -0,0 +1,31 @@
from pigway_frame_flow.models import Power
def test_observer_records_only_changes(service, monkeypatch):
driver = service.registry.get("living-room")
baseline = driver.device()
monkeypatch.setattr(driver, "device", lambda: baseline)
service.device_observer.tick()
before = len(service.db.all("SELECT * FROM events"))
service.device_observer.tick()
assert len(service.db.all("SELECT * FROM events")) == before
changed = baseline.model_copy(
update={"status": "offline", "power": Power(battery_percent=10, charging=False)}
)
monkeypatch.setattr(driver, "device", lambda: changed)
service.device_observer.tick()
assert {r["event_type"] for r in service.db.all("SELECT * FROM events")} >= {
"device.offline",
"device.low_battery",
}
def test_cloud_failure_is_not_device_offline(service, monkeypatch):
driver = service.registry.get("living-room")
def fail():
raise RuntimeError("cloud failure")
monkeypatch.setattr(driver, "device", fail)
service.device_observer.tick()
assert not service.db.all("SELECT * FROM events")
+60
View File
@@ -0,0 +1,60 @@
from datetime import UTC, datetime
import pytest
from pigway_frame_flow.errors import FlowError
from pigway_frame_flow.providers.ical import calendar_events
def feed(properties):
return (
"BEGIN:VCALENDAR\r\nBEGIN:VEVENT\r\n"
+ properties
+ "\r\nEND:VEVENT\r\nEND:VCALENDAR"
).encode()
def test_recurring_event_exclusions_and_folded_title():
result = calendar_events(
feed(
"UID:one\r\nDTSTART:20261008T080000Z\r\nRRULE:FREQ=DAILY;COUNT=3\r\nEXDATE:20261009T080000Z\r\nSUMMARY:Family\r\n time"
),
datetime(2026, 10, 8, tzinfo=UTC),
)
assert len(result["events"]) == 2
assert result["events"][0]["title"] == "Family time"
assert result["events"][1]["start"].startswith("2026-10-10")
def test_cancelled_event_ignored():
assert (
calendar_events(
feed("DTSTART:20261008T080000Z\r\nSTATUS:CANCELLED\r\nSUMMARY:Cancelled"),
datetime(2026, 10, 8, tzinfo=UTC),
)["events"]
== []
)
def test_high_frequency_rejected():
with pytest.raises(FlowError):
calendar_events(
feed("DTSTART:20261008T080000Z\r\nRRULE:FREQ=SECONDLY\r\nSUMMARY:Bad"),
datetime(2026, 10, 8, tzinfo=UTC),
)
def test_timezone_and_past_filter():
events = calendar_events(
feed("DTSTART;TZID=Asia/Shanghai:20261008T090000\r\nSUMMARY:Meeting"),
datetime(2026, 10, 8, tzinfo=UTC),
)["events"]
assert len(events) == 1
assert events[0]["start"].endswith("+08:00")
assert (
calendar_events(
feed("DTSTART:20260101T090000Z\r\nSUMMARY:Past"),
datetime(2026, 10, 8, tzinfo=UTC),
)["events"]
== []
)
+44
View File
@@ -0,0 +1,44 @@
import json
import pytest
from pigway_frame_flow.errors import FlowError
def test_mqtt_uses_scope_and_durable_idempotency(client, service, monkeypatch):
token = client.post(
"/api/v1/clients",
json={"client_id": "mqtt", "name": "MQTT", "permissions": ["display:write"]},
).json()["token"]
monkeypatch.setenv("FRAME_FLOW_MQTT_TOKEN", token)
payload = json.dumps(
{"request_id": "stable-1", "content": {"type": "template", "template": "quote"}}
).encode()
first = service.mqtt.accept(payload)
assert service.mqtt.accept(payload) == first
assert service.registry.get("living-room").client.uploads == 0
assert service.operation(first["operation_id"])["status"] == "QUEUED"
with pytest.raises(FlowError):
service.mqtt.accept(payload, retained=True)
def test_mqtt_needs_authentication(service, monkeypatch):
monkeypatch.delenv("FRAME_FLOW_MQTT_TOKEN", raising=False)
with pytest.raises(FlowError):
service.mqtt.accept(b"{}")
def test_mqtt_disabled_by_default(client, service):
assert client.get("/api/v1/integrations/mqtt").json()["config"]["enabled"] is False
service.mqtt.start()
assert service.mqtt.client is None
def test_mqtt_refuses_unapproved_plaintext(client):
assert (
client.put(
"/api/v1/integrations/mqtt",
json={"enabled": True, "host": "192.0.2.10", "tls": False},
).status_code
== 400
)
+79
View File
@@ -0,0 +1,79 @@
import json
import time
import pytest
from pigway_frame_flow.errors import FlowError
from pigway_frame_flow.providers.oauth import OAuthTokens
@pytest.mark.parametrize("kind", ["google-drive", "alipan", "baidu"])
def test_refresh_fixed_endpoint_private_persistence(
service, monkeypatch, tmp_path, kind
):
calls = []
def fetch(url, **kwargs):
calls.append((url, kwargs))
return json.dumps(
{
"access_token": "new-private-token",
"refresh_token": "rotated-private-token",
"expires_in": 3600,
}
).encode()
monkeypatch.setattr(service.assets, "fetch_url", fetch)
credentials = {
"access_token": "old-token",
"refresh_token": "refresh-private",
"client_id": "app",
"client_secret": "client-private",
}
path = tmp_path / "credentials.secret"
result = OAuthTokens(service.settings, service.assets).refresh(
kind, credentials, path
)
assert calls[0][0] == OAuthTokens.ENDPOINTS[kind]
assert result["expires_at"] > time.time() + 3500
assert json.loads(path.read_text())["refresh_token"] == "rotated-private-token"
assert path.stat().st_mode & 0o777 == 0o600
assert service.settings.sanitize("new-private-token") != "new-private-token"
def test_refresh_failure_never_retries_or_exposes_secret(service, monkeypatch):
calls = []
def fetch(*args, **kwargs):
calls.append(args)
raise RuntimeError("private-secret")
monkeypatch.setattr(service.assets, "fetch_url", fetch)
with pytest.raises(FlowError) as exc:
OAuthTokens(service.settings, service.assets).refresh(
"baidu",
{
"refresh_token": "refresh",
"client_id": "app",
"client_secret": "private-secret",
},
)
assert len(calls) == 1
assert "private-secret" not in str(exc.value)
def test_invalid_refresh_does_not_replace_existing_credentials(
service, monkeypatch, tmp_path
):
path = tmp_path / "secret"
path.write_text("original")
monkeypatch.setattr(
service.assets, "fetch_url", lambda *a, **k: b'{"error":"invalid_grant"}'
)
with pytest.raises(FlowError):
OAuthTokens(service.settings, service.assets).refresh(
"google-drive",
{"refresh_token": "refresh", "client_id": "app", "client_secret": "secret"},
path,
)
assert path.read_text() == "original"