mcpctl/.taskmaster/tasks/task_017.md

# Task ID: 17

**Title:** Implement Kubernetes Support Architecture

**Status:** pending

**Dependencies:** 6, 16

**Priority:** low

**Description:** Design and implement the abstraction layer for Kubernetes deployment support, preparing for future pod scheduling of MCP instances.

**Details:**

Create orchestrator abstraction:

```typescript
// services/orchestrator.ts
export interface McpOrchestrator {
  startServer(server: McpServer, config: any): Promise<string>;
  stopServer(instanceId: string): Promise<void>;
  getStatus(instanceId: string): Promise<InstanceStatus>;
  getLogs(instanceId: string, options: LogOptions): Promise<string>;
  listInstances(filters?: InstanceFilters): Promise<Instance[]>;
}

// Docker implementation (current)
export class DockerOrchestrator implements McpOrchestrator {
  private docker: Docker;
  // ... existing Docker implementation
}

// Kubernetes implementation (future-ready)
export class KubernetesOrchestrator implements McpOrchestrator {
  private k8sClient: KubernetesClient;

  constructor(config: K8sConfig) {
    this.k8sClient = new KubernetesClient(config);
  }

  async startServer(server: McpServer, config: any): Promise<string> {
    const pod = {
      apiVersion: 'v1',
      kind: 'Pod',
      metadata: {
        name: `mcp-${server.name}-${Date.now()}`,
        labels: {
          'mcpctl.io/server': server.name,
          'mcpctl.io/managed': 'true'
        }
      },
      spec: {
        containers: [{
          name: 'mcp-server',
          image: server.image || 'node:20-alpine',
          command: this.buildCommand(server),
          env: this.buildEnvVars(config),
          resources: {
            requests: { memory: '128Mi', cpu: '100m' },
            limits: { memory: '512Mi', cpu: '500m' }
          }
        }],
        restartPolicy: 'Always'
      }
    };

    const created = await this.k8sClient.createPod(pod);
    return created.metadata.name;
  }

  // ... other K8s implementations
}

// Factory based on configuration
export function createOrchestrator(config: OrchestratorConfig): McpOrchestrator {
  switch (config.type) {
    case 'docker': return new DockerOrchestrator(config);
    case 'kubernetes': return new KubernetesOrchestrator(config);
    default: throw new Error(`Unknown orchestrator: ${config.type}`);
  }
}
```

Configuration:
```yaml
orchestrator:
  type: docker  # or 'kubernetes'
  docker:
    socketPath: /var/run/docker.sock
  kubernetes:
    namespace: mcpctl
    kubeconfig: /path/to/kubeconfig
```

**Test Strategy:**

Unit test orchestrator interface compliance for both implementations. Integration test Docker implementation. Mock Kubernetes API for K8s implementation tests.

## Subtasks

### 17.1. Define K8s-specific interfaces and write TDD tests for KubernetesOrchestrator

**Status:** pending  
**Dependencies:** None  

Extend the McpOrchestrator interface (from Task 6) with Kubernetes-specific types and write comprehensive Vitest unit tests for all KubernetesOrchestrator methods BEFORE implementation using mocked @kubernetes/client-node.

**Details:**

Create src/shared/src/types/kubernetes.ts with K8s-specific types:

```typescript
import { McpOrchestrator, McpServer, InstanceStatus, LogOptions, InstanceFilters } from './orchestrator';

export interface K8sConfig {
  namespace: string;
  kubeconfig?: string;  // Path to kubeconfig file
  inCluster?: boolean;  // Use in-cluster config
  context?: string;     // Specific kubeconfig context
}

export interface K8sPodMetadata {
  name: string;
  namespace: string;
  labels: Record<string, string>;
  annotations: Record<string, string>;
  uid: string;
}

export interface K8sResourceRequirements {
  requests: { memory: string; cpu: string };
  limits: { memory: string; cpu: string };
}

export interface K8sSecurityContext {
  runAsNonRoot: boolean;
  runAsUser: number;
  readOnlyRootFilesystem: boolean;
  allowPrivilegeEscalation: boolean;
  capabilities: { drop: string[] };
}
```

Create src/mcpd/tests/unit/services/kubernetes-orchestrator.test.ts with comprehensive TDD tests:

1. Constructor tests: verify kubeconfig loading (file path vs in-cluster), namespace validation, error handling for missing config
2. startServer() tests: verify Pod spec generation includes security context, resource limits, labels, command building, env vars
3. stopServer() tests: verify graceful pod termination, wait for completion, error handling for non-existent pods
4. getStatus() tests: verify status mapping from K8s pod phases (Pending, Running, Succeeded, Failed, Unknown) to InstanceStatus
5. getLogs() tests: verify log options (tail, follow, since, timestamps) are mapped correctly to K8s log API
6. listInstances() tests: verify label selector filtering works, pagination handling for large deployments

Mock @kubernetes/client-node CoreV1Api using vitest.mock() with proper type definitions. All tests should fail initially (TDD red phase).

### 17.2. Implement KubernetesOrchestrator class with Pod security contexts and resource management

**Status:** pending  
**Dependencies:** 17.1  

Implement the KubernetesOrchestrator class using @kubernetes/client-node, with all methods passing TDD tests from subtask 1, including SRE-approved pod security contexts, resource requests/limits, and proper label conventions.

**Details:**

Install @kubernetes/client-node in src/mcpd. Create src/mcpd/src/services/kubernetes-orchestrator.ts:

```typescript
import * as k8s from '@kubernetes/client-node';
import { McpOrchestrator, McpServer, InstanceStatus, LogOptions, InstanceFilters, Instance } from '@mcpctl/shared';
import { K8sConfig, K8sSecurityContext, K8sResourceRequirements } from '@mcpctl/shared';

export class KubernetesOrchestrator implements McpOrchestrator {
  private coreApi: k8s.CoreV1Api;
  private namespace: string;

  constructor(config: K8sConfig) {
    const kc = new k8s.KubeConfig();
    if (config.inCluster) {
      kc.loadFromCluster();
    } else if (config.kubeconfig) {
      kc.loadFromFile(config.kubeconfig);
    } else {
      kc.loadFromDefault();
    }
    if (config.context) kc.setCurrentContext(config.context);
    this.coreApi = kc.makeApiClient(k8s.CoreV1Api);
    this.namespace = config.namespace;
  }

  async startServer(server: McpServer, config: any): Promise<string> {
    const podName = `mcp-${server.name}-${Date.now()}`;
    const pod: k8s.V1Pod = {
      apiVersion: 'v1',
      kind: 'Pod',
      metadata: {
        name: podName,
        namespace: this.namespace,
        labels: {
          'mcpctl.io/server': server.name,
          'mcpctl.io/managed': 'true',
          'app.kubernetes.io/name': `mcp-${server.name}`,
          'app.kubernetes.io/component': 'mcp-server',
          'app.kubernetes.io/managed-by': 'mcpctl'
        },
        annotations: {
          'mcpctl.io/created-at': new Date().toISOString()
        }
      },
      spec: {
        containers: [{
          name: 'mcp-server',
          image: server.image || 'node:20-alpine',
          command: this.buildCommand(server),
          env: this.buildEnvVars(config),
          resources: this.getResourceRequirements(config),
          securityContext: this.getSecurityContext()
        }],
        securityContext: { runAsNonRoot: true, runAsUser: 1000, fsGroup: 1000 },
        restartPolicy: 'Always',
        serviceAccountName: config.serviceAccount || 'default'
      }
    };
    const created = await this.coreApi.createNamespacedPod(this.namespace, pod);
    return created.body.metadata!.name!;
  }

  private getSecurityContext(): k8s.V1SecurityContext {
    return {
      runAsNonRoot: true,
      runAsUser: 1000,
      readOnlyRootFilesystem: true,
      allowPrivilegeEscalation: false,
      capabilities: { drop: ['ALL'] }
    };
  }

  private getResourceRequirements(config: any): k8s.V1ResourceRequirements {
    return {
      requests: { memory: config.memoryRequest || '128Mi', cpu: config.cpuRequest || '100m' },
      limits: { memory: config.memoryLimit || '512Mi', cpu: config.cpuLimit || '500m' }
    };
  }
  // ... implement stopServer, getStatus, getLogs, listInstances
}
```

Implement all remaining methods with proper error handling and K8s API error translation.

### 17.3. Implement createOrchestrator factory function and configuration schema

**Status:** pending  
**Dependencies:** 17.2  

Create the orchestrator factory function that instantiates DockerOrchestrator or KubernetesOrchestrator based on configuration, with Zod schema validation and configuration file support.

**Details:**

Create src/mcpd/src/services/orchestrator-factory.ts:

```typescript
import { z } from 'zod';
import { McpOrchestrator } from '@mcpctl/shared';
import { DockerOrchestrator } from './container-manager'; // From Task 6
import { KubernetesOrchestrator } from './kubernetes-orchestrator';

const DockerConfigSchema = z.object({
  socketPath: z.string().default('/var/run/docker.sock'),
  host: z.string().optional(),
  port: z.number().optional(),
  network: z.string().default('mcpctl-network')
});

const KubernetesConfigSchema = z.object({
  namespace: z.string().default('mcpctl'),
  kubeconfig: z.string().optional(),
  inCluster: z.boolean().default(false),
  context: z.string().optional()
});

const OrchestratorConfigSchema = z.discriminatedUnion('type', [
  z.object({ type: z.literal('docker'), docker: DockerConfigSchema }),
  z.object({ type: z.literal('kubernetes'), kubernetes: KubernetesConfigSchema })
]);

export type OrchestratorConfig = z.infer<typeof OrchestratorConfigSchema>;

export function createOrchestrator(config: OrchestratorConfig): McpOrchestrator {
  const validated = OrchestratorConfigSchema.parse(config);
  switch (validated.type) {
    case 'docker':
      return new DockerOrchestrator(validated.docker);
    case 'kubernetes':
      return new KubernetesOrchestrator(validated.kubernetes);
    default:
      throw new Error(`Unknown orchestrator type`);
  }
}
```

Create src/mcpd/src/config/orchestrator.ts for loading config from environment variables and config files (supporting both YAML and JSON). Write TDD tests in src/mcpd/tests/unit/services/orchestrator-factory.test.ts BEFORE implementation:

1. Test factory creates DockerOrchestrator when type='docker'
2. Test factory creates KubernetesOrchestrator when type='kubernetes'
3. Test factory throws on invalid type
4. Test Zod validation rejects invalid configs
5. Test default values are applied correctly
6. Test config loading from MCPCTL_ORCHESTRATOR_TYPE env var

### 17.4. Implement K8s NetworkPolicy and PersistentVolumeClaim builders for MCP server isolation

**Status:** pending  
**Dependencies:** 17.2  

Create resource builders for Kubernetes NetworkPolicy (network isolation between MCP servers) and PersistentVolumeClaim (for stateful data MCPs like caching or GPU providers) with proper annotations for observability.

**Details:**

Create src/mcpd/src/services/k8s-resources.ts with resource builder functions:

```typescript
import * as k8s from '@kubernetes/client-node';

export interface NetworkPolicyConfig {
  serverName: string;
  namespace: string;
  allowEgress?: string[];  // CIDR blocks or service names to allow
  allowIngress?: string[]; // Pod labels allowed to connect
}

export function buildNetworkPolicy(config: NetworkPolicyConfig): k8s.V1NetworkPolicy {
  return {
    apiVersion: 'networking.k8s.io/v1',
    kind: 'NetworkPolicy',
    metadata: {
      name: `mcp-${config.serverName}-netpol`,
      namespace: config.namespace,
      labels: { 'mcpctl.io/server': config.serverName, 'mcpctl.io/managed': 'true' }
    },
    spec: {
      podSelector: { matchLabels: { 'mcpctl.io/server': config.serverName } },
      policyTypes: ['Ingress', 'Egress'],
      ingress: [{
        from: [{ podSelector: { matchLabels: { 'mcpctl.io/component': 'local-proxy' } } }]
      }],
      egress: config.allowEgress?.map(cidr => ({
        to: [{ ipBlock: { cidr } }]
      })) || [{ to: [{ ipBlock: { cidr: '0.0.0.0/0' } }] }]  // Default: allow all egress
    }
  };
}

export interface PVCConfig {
  serverName: string;
  namespace: string;
  storageSize: string;  // e.g., '1Gi'
  storageClass?: string;
  accessModes?: string[];
}

export function buildPVC(config: PVCConfig): k8s.V1PersistentVolumeClaim {
  return {
    apiVersion: 'v1',
    kind: 'PersistentVolumeClaim',
    metadata: {
      name: `mcp-${config.serverName}-data`,
      namespace: config.namespace,
      labels: { 'mcpctl.io/server': config.serverName, 'mcpctl.io/managed': 'true' },
      annotations: {
        'mcpctl.io/purpose': 'mcp-server-cache',
        'mcpctl.io/created-at': new Date().toISOString()
      }
    },
    spec: {
      accessModes: config.accessModes || ['ReadWriteOnce'],
      storageClassName: config.storageClass,
      resources: { requests: { storage: config.storageSize } }
    }
  };
}

export function buildGpuAffinityRules(gpuType: string): k8s.V1Affinity {
  return {
    nodeAffinity: {
      requiredDuringSchedulingIgnoredDuringExecution: {
        nodeSelectorTerms: [{
          matchExpressions: [{
            key: 'nvidia.com/gpu.product',
            operator: 'In',
            values: [gpuType]
          }]
        }]
      }
    }
  };
}
```

Write TDD tests in src/mcpd/tests/unit/services/k8s-resources.test.ts verifying all resource builders generate valid K8s manifests.

### 17.5. Create integration tests with kind/k3d and document K8s deployment architecture

**Status:** pending  
**Dependencies:** 17.2, 17.3, 17.4  

Build integration test suite using kind or k3d for local K8s cluster testing, create comprehensive SRE documentation covering deployment architecture, resource recommendations, and network requirements.

**Details:**

Create src/mcpd/tests/integration/kubernetes/ directory with integration tests:

1. Create setup script src/mcpd/tests/integration/kubernetes/setup-kind.ts:
```typescript
import { execSync } from 'child_process';

export async function setupKindCluster(): Promise<void> {
  execSync('kind create cluster --name mcpctl-test --config tests/integration/kubernetes/kind-config.yaml', { stdio: 'inherit' });
}

export async function teardownKindCluster(): Promise<void> {
  execSync('kind delete cluster --name mcpctl-test', { stdio: 'inherit' });
}
```

2. Create kind-config.yaml with proper resource limits
3. Create kubernetes-orchestrator.integration.test.ts testing:
   - Pod creation and deletion lifecycle
   - Status monitoring through pod phases
   - Log retrieval from running pods
   - NetworkPolicy enforcement (cannot reach blocked endpoints)
   - PVC mounting for stateful MCPs

4. Create src/mcpd/docs/KUBERNETES_DEPLOYMENT.md documenting:
   - Architecture overview: mcpctl namespace, resource types, label conventions
   - Security: Pod security standards (restricted), NetworkPolicies, ServiceAccounts
   - SRE recommendations: HPA configurations, PDB templates, monitoring with Prometheus labels
   - Resource sizing guide: Small (128Mi/100m), Medium (512Mi/500m), Large (2Gi/1000m)
   - Network requirements: Required egress rules per MCP server type, ingress from local-proxy
   - Troubleshooting: Common issues, kubectl commands, log access
   - GPU support: Node affinity, NVIDIA device plugin requirements

5. Create example manifests in src/mcpd/examples/k8s/:
   - namespace.yaml, rbac.yaml, networkpolicy.yaml, sample-mcp-pod.yaml

Integration tests should skip gracefully when kind is not available (CI compatibility).