Align project introduction with bilingual PIGWay READMEs
This commit is contained in:
@@ -0,0 +1,103 @@
|
||||
# Build networking, cache and CI
|
||||
|
||||
### Build networking and cache
|
||||
|
||||
The Dockerfile retains the pinned Python base, locked Python dependencies,
|
||||
`fonts-noto-cjk`, Debian repository signature checks, and offline artifact
|
||||
SHA256 verification. Production uses `scripts/build-verify.sh`; CI uses `scripts/ci-compose-verify.py`.
|
||||
Both build the same Dockerfile with BuildKit and retain `docker compose build --pull`.
|
||||
|
||||
APT mirror hosts are configurable through `DEBIAN_MIRROR` and
|
||||
`DEBIAN_SECURITY_MIRROR` (hosts only, without scheme or path). Defaults are
|
||||
`deb.debian.org`; HTTPS is used for both Debian and security repositories.
|
||||
For mainland-China NAS builds, use the shared profile:
|
||||
|
||||
```sh
|
||||
. ./deploy/build-cn.env
|
||||
./scripts/build-verify.sh
|
||||
```
|
||||
|
||||
CI loads this same profile, using Tsinghua TUNA for both repositories unless
|
||||
Gitea repository variables `DEBIAN_MIRROR` / `DEBIAN_SECURITY_MIRROR` override
|
||||
it. Production may also set the two hosts directly in `.env` when calling the
|
||||
build script without the profile. An alternate mainland mirror is
|
||||
`mirrors.ustc.edu.cn`; availability and throughput depend on the Runner network.
|
||||
No fallback disables signature verification or removes packages.
|
||||
|
||||
Layer order stays: pinned base → APT/font dependencies → `requirements.lock`
|
||||
→ pip dependencies → application/vendor source → build metadata.
|
||||
A source-only change therefore reuses the APT and pip layers, provided the
|
||||
same builder, base digest, mirror arguments, and optional offline artifacts
|
||||
are used. Changing the lockfile rebuilds pip and later layers, not APT.
|
||||
Changing the mirror or APT instructions rebuilds APT and later layers.
|
||||
|
||||
Previously an APT layer miss redownloaded indexes/packages, and pip used
|
||||
`--no-cache-dir`. Now locked BuildKit mounts retain APT indexes, downloaded
|
||||
`.deb` files, and pip downloads/wheels outside the final image. Debian's
|
||||
`docker-clean` hook is disabled so it cannot discard cached packages.
|
||||
Re-executed APT steps still run `apt-get update` and validate repository
|
||||
signatures; cached packages and pip artifacts can avoid repeat large downloads.
|
||||
The initial build after this Dockerfile change will miss dependency layers;
|
||||
subsequent builds benefit. No measured speedup is claimed before CI completes.
|
||||
|
||||
Caches belong to the Docker builder, not the ephemeral CI workspace. Keep the
|
||||
NAS Runner's builder storage across jobs; removing the builder, pruning caches,
|
||||
or using a fresh Docker daemon loses the benefit. Cache mounts are disposable:
|
||||
an empty cache still builds correctly. `--pull` remains enabled; a pinned base
|
||||
digest check alone does not invalidate dependency layers. To deliberately
|
||||
refresh installed APT versions, use `docker compose build --pull --no-cache`.
|
||||
|
||||
### CI with a host Docker socket
|
||||
|
||||
Run `python scripts/ci-compose-verify.py` after loading the build mirror profile.
|
||||
The script derives a temporary Compose document from production settings, with
|
||||
an absolute client-side build context, a UUID project name, a unique CI image
|
||||
tag, and project-scoped `data` / `config` named volumes. It removes production
|
||||
bind mounts, the fixed container name, host ports, and the production `.env`
|
||||
reference. A fresh test token, mock driver, and disabled device writes are
|
||||
explicitly configured. Docker initializes empty named volumes from the image's
|
||||
owned `/data` and `/config` directories; the service remains UID/GID 10333.
|
||||
|
||||
Build contexts are sent by the Docker client, so the job workspace can remain
|
||||
inside the job container. Runtime storage is managed by the daemon; no matching
|
||||
NAS host `/workspace` directory is needed. HTTP readiness and the unmodified
|
||||
nine-endpoint smoke script run inside the service with `compose exec -T`.
|
||||
Restart is followed by another readiness check and the same smoke test.
|
||||
|
||||
A `finally` block removes the test containers, network, volumes, and unique
|
||||
image tag on success or failure. The workflow records the private temporary
|
||||
Compose path in `GITHUB_ENV` and repeats cleanup in an `always()` step, covering
|
||||
normal cancellation/interruption. Failed cleanup retains the manifest so the
|
||||
fallback can retry. Abrupt host/daemon loss or a forcibly terminated entire job
|
||||
can prevent both cleanup paths; no CI script can guarantee cleanup in that case.
|
||||
No global prune or production resource removal is performed. Keep builder
|
||||
caches: only the run-specific image tag is removed.
|
||||
|
||||
### CI Job pip indexes versus Docker build indexes
|
||||
|
||||
The Job bootstrap runs `python scripts/ci-install.py` before Ruff/mypy/tests.
|
||||
It needs only the Python standard library, so it runs before PyYAML or tooling
|
||||
is installed. `CI_PIP_INDEX_URL` and `CI_PIP_FALLBACK_INDEX_URL` are optional
|
||||
Gitea repository variables. Defaults are the TUNA HTTPS PyPI simple endpoint
|
||||
and official `https://pypi.org/simple`. Each is attempted separately; the
|
||||
fallback repeats the unchanged editable `.[test]` installation, including
|
||||
PEP 517 build isolation and `setuptools>=75`. No simultaneous extra index is
|
||||
used, no build dependency is removed, and both failures fail CI.
|
||||
|
||||
Logs show inherited index/config/proxy hosts and the selected index host for
|
||||
each attempt, with URL credentials and query strings removed. Bootstrap
|
||||
disables inherited pip config files via `PIP_CONFIG_FILE=/dev/null`, explicitly
|
||||
sets the selected index, and clears no-index/extra-index/find-links, insecure
|
||||
trusted-host, and no-build-isolation environment settings. Constraints, TLS
|
||||
certificate environment settings, and proxy settings remain in effect.
|
||||
|
||||
These `CI_PIP_*` settings apply only to the Job's pip install. They are not
|
||||
passed into Docker build arguments or persisted into the production image.
|
||||
The Dockerfile keeps its own cached/offline dependency installation and logs
|
||||
the Docker pip index host for online installs. Debian mirror variables affect
|
||||
APT only and are loaded later in the Docker verification step.
|
||||
|
||||
If dependency installation fails, the always() cleanup step still executes.
|
||||
It skips the Python/PyYAML cleanup script when no CI Compose manifest exists,
|
||||
as no Docker verification resources have been created at that point.
|
||||
|
||||
Reference in New Issue
Block a user