182 lines
11 KiB
Markdown
182 lines
11 KiB
Markdown
# Task ID: 6
|
|
|
|
**Title:** Implement Docker Container Management for MCP Servers
|
|
|
|
**Status:** pending
|
|
|
|
**Dependencies:** 3, 4
|
|
|
|
**Priority:** high
|
|
|
|
**Description:** Create the container orchestration layer for running MCP servers as Docker containers, with support for docker-compose deployment.
|
|
|
|
**Details:**
|
|
|
|
Create Docker management module:
|
|
|
|
```typescript
|
|
// services/container-manager.ts
|
|
import Docker from 'dockerode';
|
|
|
|
export class ContainerManager {
|
|
private docker: Docker;
|
|
|
|
constructor() {
|
|
this.docker = new Docker({ socketPath: '/var/run/docker.sock' });
|
|
}
|
|
|
|
async startMcpServer(server: McpServer, config: McpProfile['config']): Promise<string> {
|
|
const container = await this.docker.createContainer({
|
|
Image: server.image || 'node:20-alpine',
|
|
Cmd: this.buildCommand(server, config),
|
|
Env: this.buildEnvVars(server, config),
|
|
Labels: {
|
|
'mcpctl.server': server.name,
|
|
'mcpctl.managed': 'true'
|
|
},
|
|
HostConfig: {
|
|
NetworkMode: 'mcpctl-network',
|
|
RestartPolicy: { Name: 'unless-stopped' }
|
|
}
|
|
});
|
|
await container.start();
|
|
return container.id;
|
|
}
|
|
|
|
async stopMcpServer(containerId: string): Promise<void> {
|
|
const container = this.docker.getContainer(containerId);
|
|
await container.stop();
|
|
await container.remove();
|
|
}
|
|
|
|
async getMcpServerStatus(containerId: string): Promise<'running' | 'stopped' | 'error'> {
|
|
try {
|
|
const container = this.docker.getContainer(containerId);
|
|
const info = await container.inspect();
|
|
return info.State.Running ? 'running' : 'stopped';
|
|
} catch {
|
|
return 'error';
|
|
}
|
|
}
|
|
|
|
async listManagedContainers(): Promise<Docker.ContainerInfo[]> {
|
|
return this.docker.listContainers({
|
|
filters: { label: ['mcpctl.managed=true'] }
|
|
});
|
|
}
|
|
}
|
|
```
|
|
|
|
Create docker-compose.yml template:
|
|
```yaml
|
|
version: '3.8'
|
|
services:
|
|
mcpd:
|
|
build: ./src/mcpd
|
|
ports:
|
|
- "3000:3000"
|
|
environment:
|
|
- DATABASE_URL=postgresql://...
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
networks:
|
|
- mcpctl-network
|
|
|
|
postgres:
|
|
image: postgres:15
|
|
volumes:
|
|
- pgdata:/var/lib/postgresql/data
|
|
networks:
|
|
- mcpctl-network
|
|
|
|
networks:
|
|
mcpctl-network:
|
|
driver: bridge
|
|
|
|
volumes:
|
|
pgdata:
|
|
```
|
|
|
|
**Test Strategy:**
|
|
|
|
Test container creation, start, stop, and removal. Test status checking. Integration test with actual Docker daemon. Verify network isolation works correctly.
|
|
|
|
## Subtasks
|
|
|
|
### 6.1. Define McpOrchestrator interface and write TDD tests for ContainerManager
|
|
|
|
**Status:** pending
|
|
**Dependencies:** None
|
|
|
|
Define the McpOrchestrator abstraction interface that both DockerOrchestrator (this task) and KubernetesOrchestrator (task 17) will implement. Write comprehensive Vitest unit tests for all ContainerManager methods BEFORE implementation using dockerode mocks.
|
|
|
|
**Details:**
|
|
|
|
Create src/mcpd/src/services/orchestrator.ts with the McpOrchestrator interface including: startServer(), stopServer(), getStatus(), getLogs(), listInstances(). Then create src/mcpd/src/services/docker/__tests__/container-manager.test.ts with TDD tests covering: (1) constructor connects to Docker socket, (2) startMcpServer() creates container with correct labels, env vars, and network config, (3) stopMcpServer() stops and removes container, (4) getMcpServerStatus() returns 'running', 'stopped', or 'error' states, (5) listManagedContainers() filters by mcpctl.managed label, (6) buildCommand() generates correct command array from server config, (7) buildEnvVars() maps profile config to environment variables. Use vi.mock('dockerode') to mock all Docker operations. Tests should initially fail (TDD red phase).
|
|
|
|
### 6.2. Implement ContainerManager class with DockerOrchestrator strategy pattern
|
|
|
|
**Status:** pending
|
|
**Dependencies:** 6.1
|
|
|
|
Implement the ContainerManager class as a DockerOrchestrator implementation using dockerode, with all methods passing the TDD tests from subtask 1.
|
|
|
|
**Details:**
|
|
|
|
Create src/mcpd/src/services/docker/container-manager.ts implementing McpOrchestrator interface. Constructor accepts optional Docker socket path (default: /var/run/docker.sock). Implement startMcpServer(): create container with Image (server.image || 'node:20-alpine'), Cmd from buildCommand(), Env from buildEnvVars(), Labels (mcpctl.server, mcpctl.managed, mcpctl.profile), HostConfig with NetworkMode 'mcpctl-network' and RestartPolicy 'unless-stopped'. Implement stopMcpServer(): stop() then remove() the container. Implement getMcpServerStatus(): inspect() container and return state. Implement listManagedContainers(): listContainers() with label filter. Implement buildCommand(): parse server.command template with config substitutions. Implement buildEnvVars(): merge server.envTemplate with profile.config values. Add resource limits to HostConfig (Memory: 512MB default, NanoCPUs: 1e9 default) - these are overridable via server config. All TDD tests from subtask 1 should now pass.
|
|
|
|
### 6.3. Create docker-compose.yml template with mcpd, PostgreSQL, and test MCP server
|
|
|
|
**Status:** pending
|
|
**Dependencies:** None
|
|
|
|
Create the production-ready docker-compose.yml template for local development with mcpd service, PostgreSQL database, a test MCP server container, and proper networking configuration.
|
|
|
|
**Details:**
|
|
|
|
Create deploy/docker-compose.yml with services: (1) mcpd - build from src/mcpd, expose port 3000, DATABASE_URL env var, mount /var/run/docker.sock (read-only), depends_on postgres with healthcheck, deploy resources limits (memory: 512M), restart: unless-stopped. (2) postgres - postgres:15-alpine image, POSTGRES_USER/PASSWORD/DB env vars, healthcheck with pg_isready, volume for pgdata, deploy resources limits (memory: 256M). (3) test-mcp-server - simple echo server image (node:20-alpine with npx @modelcontextprotocol/server-memory), labels for mcpctl.managed and mcpctl.server, same network. Create mcpctl-network as bridge driver. Create named volumes: pgdata. Add .env.example with required environment variables. Ensure all containers have resource limits and no --privileged flag. Add docker-compose.test.yml override for CI testing with ephemeral volumes.
|
|
|
|
### 6.4. Write integration tests with real Docker daemon
|
|
|
|
**Status:** pending
|
|
**Dependencies:** 6.2, 6.3
|
|
|
|
Create integration test suite that tests ContainerManager against a real Docker daemon, verifying actual container lifecycle operations work correctly.
|
|
|
|
**Details:**
|
|
|
|
Create src/mcpd/src/services/docker/__tests__/container-manager.integration.test.ts. Use vitest with longer timeout (30s). Before all: ensure mcpctl-network exists (create if not). After each: cleanup any test containers. Test cases: (1) startMcpServer() creates a real container with test MCP server image, verify container is running with docker inspect, (2) getMcpServerStatus() returns 'running' for active container, (3) stopMcpServer() removes container and getMcpServerStatus() returns 'error', (4) listManagedContainers() returns only containers with mcpctl.managed label, (5) test container networking - two MCP server containers can communicate on mcpctl-network. Use node:20-alpine with simple sleep command as test image. Add CI skip condition (describe.skipIf(!process.env.DOCKER_HOST)) for environments without Docker. Tag tests with '@integration' for selective running.
|
|
|
|
### 6.5. Implement container network isolation and resource management
|
|
|
|
**Status:** pending
|
|
**Dependencies:** 6.2
|
|
|
|
Add network segmentation utilities and resource management capabilities to ensure proper isolation between MCP server containers and prevent resource exhaustion.
|
|
|
|
**Details:**
|
|
|
|
Create src/mcpd/src/services/docker/network-manager.ts with: ensureNetworkExists() - creates mcpctl-network if not present with bridge driver, getNetworkInfo() - returns network details, connectContainer() - adds container to network, disconnectContainer() - removes from network. Add to ContainerManager: getContainerStats() - returns CPU/memory usage via container.stats(), setResourceLimits() - updates container resources. Implement container isolation: each MCP server profile can specify allowed networks, default deny all external network access, only allow container-to-container on mcpctl-network. Add ResourceConfig type with memory (bytes), cpuShares, cpuPeriod, pidsLimit. Write unit tests for network-manager with mocked dockerode. Integration test: start two containers, verify they can reach each other on mcpctl-network but not external network.
|
|
|
|
### 6.6. Conduct security review of Docker socket access and container configuration
|
|
|
|
**Status:** pending
|
|
**Dependencies:** 6.2, 6.3, 6.5
|
|
|
|
Perform comprehensive security review of all Docker-related code, documenting risks of Docker socket access and implementing security controls for container isolation.
|
|
|
|
**Details:**
|
|
|
|
Create src/mcpd/docs/DOCKER_SECURITY_REVIEW.md documenting: (1) Docker socket access risks - socket access grants root-equivalent privileges, mitigations implemented (read-only mount where possible, no container creation with --privileged, no host network mode, no host PID namespace). (2) Container escape prevention - no --privileged containers, no SYS_ADMIN capability, seccomp profile enabled (default), AppArmor profile enabled (default), drop all capabilities except required ones. (3) Image source validation - add validateImageSource() function that checks image against allowlist, reject images from untrusted registries, warn on :latest tags. (4) Resource limits - all containers MUST have memory and CPU limits, pids-limit to prevent fork bombs. (5) Network segmentation - MCP servers isolated to mcpctl-network, no external network access by default. (6) Secrets handling - environment variables with credentials are passed at runtime not build time, no secrets in image layers. Add security tests that verify: no --privileged, caps are dropped, resource limits are set.
|
|
|
|
### 6.7. Implement container logs streaming and health monitoring
|
|
|
|
**Status:** pending
|
|
**Dependencies:** 6.2
|
|
|
|
Add log streaming capabilities and health monitoring to ContainerManager to support instance lifecycle management (Task 16) and provide observability into running MCP servers.
|
|
|
|
**Details:**
|
|
|
|
Extend ContainerManager with: getLogs(containerId, options: LogOptions): AsyncIterator<string> - streams logs from container using dockerode container.logs() with follow option, LogOptions includes timestamps, tail lines count, since timestamp. getHealthStatus(containerId): returns health check result if container has HEALTHCHECK, otherwise infers from running state. attachToContainer(containerId): returns bidirectional stream for stdio. Add event subscriptions: onContainerStart, onContainerStop, onContainerDie callbacks using Docker events API. Create src/mcpd/src/services/docker/container-events.ts with ContainerEventEmitter class that listens to Docker daemon events and emits typed events. Write unit tests mocking dockerode stream responses. Integration test: start container, tail logs, verify log output matches container stdout. Test event subscription receives container lifecycle events.
|