Instance status now reflects actual container state:
- startOne() sets STARTING (not RUNNING) after container creation
- syncStatus() promotes STARTING→RUNNING when pod is ready
- syncStatus() demotes RUNNING→STARTING if pod restarts (CrashLoop)
- External servers still get RUNNING immediately (no container)
Previously, CrashLooping pods showed as RUNNING in mcpctl get instances.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two bugs fixed:
1. Backup completeness: JSON backup API now includes prompts and
templates. Previously these were silently dropped during
backup/restore, causing data loss on migration.
2. STDIO proxy for docker-image servers: servers with dockerImage
but no packageName/command (like docmost) now use k8s Attach
to connect to the container's PID 1 stdin/stdout instead of
exec. This fixes "has no packageName or command" errors.
Changes:
- backup-service.ts: add BackupPrompt/BackupTemplate types, export them
- restore-service.ts: restore prompts (with project FK) and templates
- mcp-proxy-service.ts: sendViaPersistentAttach for docker-image STDIO
- orchestrator.ts: add attachInteractive to McpOrchestrator interface
- kubernetes-orchestrator.ts: implement attachInteractive via k8s Attach
- k8s-client-official.ts: expose Attach client
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mcpd now runs a periodic reconcileAll() every 30s that:
- Detects crashed/missing containers (syncStatus)
- Cleans up ERROR instances
- Creates replacement pods to match desired replica count
This replaces the old syncStatus-only timer. Servers migrated
from another deployment or recovering from node failures will
automatically get their instances recreated.
6 new tests for reconcileAll covering: missing instances, skip
replicas=0, already-at-count, ERROR cleanup, multi-server,
error isolation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add MCPD_NODE_SELECTOR env var support in manifest generator
for mixed-arch clusters (e.g. arm64+amd64)
- Fix backup restore: resolve system user ID instead of
hardcoded 'system' string
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The restore service hardcoded ownerId as the literal string 'system'
instead of looking up the actual system user ID. This caused FK
constraint violations when restoring projects to a fresh database.
Now resolves the system user by email, falling back to the first
available user.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
mcpd can now deploy MCP server instances as Kubernetes pods instead of
Docker containers. Set MCPD_ORCHESTRATOR=kubernetes to enable.
- Add @kubernetes/client-node with thin wrapper (context enforcement
via MCPD_K8S_CONTEXT to prevent multi-cluster mishaps)
- Rewrite KubernetesOrchestrator: pod CRUD, pod IP extraction,
exec via SPDY (one-shot + interactive), log streaming
- Manifest generator: stdin:true for STDIO servers, args (not command)
to preserve runner image entrypoint, security hardening
- Orchestrator selection in main.ts via MCPD_ORCHESTRATOR env var
- 25 unit tests for k8s orchestrator, all 624 tests pass
Tested end-to-end on local k3s:
- mcpd deployed via Pulumi, creates pods in mcpctl-servers namespace
- NetworkPolicy verified: only mcpd can reach MCP server pods
- Python runner (uvx) successfully runs aws-documentation-mcp-server
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The act runner (v0.3.0) on NAS can't handle matrix jobs reliably on a
single worker — concurrent matrix entries fail silently. Build both
amd64 and arm64 sequentially in a single job instead.
Merge publish-rpm and publish-deb into a single publish job that
iterates over all RPM/DEB files in dist/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The publish-rpm step was deleting the existing package by version
before uploading, but Gitea RPM registry keys by version (not
version+arch). When building both amd64 and arm64 in a matrix,
the second job would delete the first job's upload.
Remove the delete-before-upload pattern. Gitea supports multiple
architectures under the same version. Handle 409 (already exists)
gracefully instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Server instances require Docker/Podman (mcpd starts them as containers).
CI has no container runtime, so instances will never reach RUNNING.
Tests requiring running instances are already excluded.
Replace the 5-minute wait loop with a quick fixture verification step
that confirms servers, projects, and prompts were applied correctly,
and reports instance status for informational purposes only.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- publish-rpm and publish-deb now depend on both build and smoke jobs,
so packages are only published after all tests pass
- Reduce "Wait for server instance" from 60x5s (5min) to 10x2s (20s)
since Docker containers can't run in CI anyway
- Add debug output to RPM/DEB packaging steps
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- nfpm.yaml: use ${NFPM_ARCH} (Go's ExpandEnv doesn't support :-default)
- arch-helper.sh: export RPM_ARCH (x86_64/aarch64) alongside NFPM_ARCH
- build-rpm/deb.sh: build TypeScript before running tests (tests need
built @mcpctl/shared), generate Prisma client on fresh checkout
- Fix RPM filename matching to use aarch64 not arm64
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Build scripts now check for required tools before building and install
them automatically if missing. Handles both amd64 and arm64 host systems.
- pnpm: installed via corepack or npm
- bun: installed via official install script
- nfpm: downloaded from GitHub for the correct host architecture
- node_modules: runs pnpm install if missing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add cross-architecture build support so the project can be developed on
ARM64 (Fedora aarch64 laptop) while still producing amd64 packages for
production. All build, package, publish, and install scripts are now
architecture-aware via shared arch-helper.sh detection.
- Add scripts/arch-helper.sh for shared architecture detection
- CI builds both amd64 and arm64 in matrix strategy
- nfpm.yaml uses NFPM_ARCH env var instead of hardcoded amd64
- Build scripts support MCPCTL_TARGET_ARCH for cross-compilation
- installlocal.sh auto-detects RPM/DEB and filters by architecture
- release.sh gains --both-arches flag for dual-arch releases
- Package cleanup is arch-scoped (won't clobber other arch's packages)
- build-mcpd.sh supports --platform and --multi-arch flags
- Add pnpm scripts: rpm:build:amd64, deb:build:arm64, release:both
- Conditional rpm/dpkg-deb checks for cross-distro compatibility
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fan-out discovery methods (tools/list, prompts/list, resources/list)
used synthetic request IDs that couldn't be looked up in the
correlation map. This caused upstream_response events to have no
correlationId, making the console unable to find upstream content
for replay ("No content to replay").
Fix: pass correlationId through RouteContext → discovery methods →
onUpstreamCall callback, so the handler can use it directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Internal API calls still use 10.0.0.194:3012, but all user-facing
install instructions now use the public GITEA_PUBLIC_URL.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Support DEB packaging alongside RPM for Debian trixie (13/stable),
forky (14/testing), Ubuntu noble (24.04 LTS), and plucky (25.04).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Configure mcplocal with anthropic (claude-haiku-3.5) in CI using
the ANTHROPIC_API_KEY secret. Writes ~/.mcpctl/config.json and
~/.mcpctl/secrets before starting mcplocal.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Run vitest with --no-file-parallelism to prevent concurrent requests
from crashing mcplocal. Also capture mcplocal output to a log file
and dump it on failure for debugging.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
These tests create MCP sessions to smoke-data which tries to proxy to
the smoke-aws-docs server container. Without Docker in CI, mcplocal
crashes when it attempts to connect to the non-existent container.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The security tests open an SSE connection to /inspect that crashes
mcplocal, cascading into timeouts for audit and proxy-pipeline tests.
They also need LLM providers not available in CI. These tests document
known vulnerabilities and work locally against production.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use `pnpm --filter @mcpctl/db exec` to run the CI user setup script
so @prisma/client resolves correctly under pnpm's strict layout.
Also remove unused bcrypt dependency.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The auth/bootstrap endpoint fails with 409 because mcpd's startup
creates a system user (system@mcpctl.local), making the "no users
exist" check fail. Instead, create the CI user, session token, and
RBAC definition directly in postgres via Prisma.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The curl -sf flag was hiding the actual HTTP error body. Now we capture
and display the full response to diagnose why auth bootstrap fails.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The single-worker Gitea runner consistently hangs when multiple parallel
jobs try to restore the pnpm cache simultaneously. Removing cache: pnpm
from setup-node trades slightly slower installs for reliable execution.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Runs in parallel with the build job after lint/typecheck/test pass.
Spins up PostgreSQL via services, bootstraps auth, starts mcpd and
mcplocal from source, applies smoke fixtures (aws-docs server + 100
prompts), and runs the full smoke test suite.
Container management for upstream MCP servers depends on Docker socket
availability in the runner — emits a warning if unavailable.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The Gitea Act Runner containers lack privileged access needed for
container-in-container builds. Tried: Docker CLI (permission denied),
podman (cannot re-exec), buildah (no /proc/self/uid_map), kaniko
(no standalone binary). Docker builds + deploy continue to work via
bash fulldeploy.sh which runs on the host directly.
CI pipeline now: lint → typecheck → test → build → publish-rpm
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Docker, podman, and buildah all fail in the runner container due to
missing /proc/self/uid_map (no user namespace support). Kaniko is
designed specifically for building Docker images inside containers
without privileged access, Docker daemon, or user namespaces.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Podman fails with "cannot re-exec process" inside runner containers
(no user namespace support). Buildah with --isolation chroot and
--storage-driver vfs can build OCI images without a daemon, without
namespaces, and without privileged mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Docker CLI can't connect to the podman socket in the runner container
(permission denied even as root). Switch to podman for building images
locally and skopeo with containers-storage transport for pushing.
Podman builds don't need a daemon socket.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The default runner image (catthehacker/ubuntu:act-latest) has the
podman socket mounted at /var/run/docker.sock but no Docker CLI.
Install docker.io to provide the CLI. The socket is accessible as
root, so sudo -E docker build works.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add debug step to understand docker socket state in runner container.
Restore sudo -E for docker/skopeo commands and remove container block
(runner already mounts podman socket by default).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bun's bundler can't read directory symlinks (EISDIR). Copy the stub
files directly into node_modules instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
shamefully-hoist still creates symlinks to .pnpm store which bun
can't follow (EISDIR errors). node-linker=hoisted creates actual
copies in a flat node_modules layout, like npm.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bun's bundler can't follow pnpm's nested symlink layout to resolve
transitive dependencies of workspace packages (e.g. ink's yoga-layout,
react-reconciler). Adding shamefully-hoist=true creates a flat
node_modules layout that bun can resolve from, matching the behavior
of the local dev environment.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The runner container doesn't have access to the Docker socket by
default. Mount /var/run/docker.sock via container.volumes so docker
build and skopeo can access the host's podman API. Removed sudo since
the container user is root.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The local build-rpm.sh successfully uses pnpm's node_modules with bun
compile. The CI was unnecessarily replacing node_modules with bun install,
which broke transitive workspace dependency resolution. Match the working
local approach instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sudo resets the environment by default, so DOCKER_API_VERSION=1.43
wasn't reaching the docker CLI. Use -E to preserve it.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Docker CLI v1.52 is too new for the host's podman daemon (max 1.43).
Set DOCKER_API_VERSION to force the older API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
bun install on top of pnpm's nested node_modules fails to resolve
workspace transitive deps (Ink, inquirer, etc). Remove node_modules
first so bun creates a proper flat layout from scratch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
docker build works via podman socket (builds don't need registry access).
skopeo pushes directly over HTTP with --dest-tls-verify=false, bypassing
the daemon's registry config entirely. No login/daemon config needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The host uses podman (not Docker) — the socket mounted in job containers
is /run/podman/podman.sock. Podman reads /etc/containers/registries.conf
for insecure registry config, which takes effect immediately without any
daemon restart.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
