first commit

2026-02-21 03:10:39 +00:00
commit d0aa0c5d63
174 changed files with 21169 additions and 0 deletions
--- a/.taskmaster/tasks/task_017.md
+++ b/.taskmaster/tasks/task_017.md
@@ -0,0 +1,443 @@
+# Task ID: 17
+
+**Title:** Implement Kubernetes Support Architecture
+
+**Status:** pending
+
+**Dependencies:** 6, 16
+
+**Priority:** low
+
+**Description:** Design and implement the abstraction layer for Kubernetes deployment support, preparing for future pod scheduling of MCP instances.
+
+**Details:**
+
+Create orchestrator abstraction:
+
+```typescript
+// services/orchestrator.ts
+export interface McpOrchestrator {
+  startServer(server: McpServer, config: any): Promise<string>;
+  stopServer(instanceId: string): Promise<void>;
+  getStatus(instanceId: string): Promise<InstanceStatus>;
+  getLogs(instanceId: string, options: LogOptions): Promise<string>;
+  listInstances(filters?: InstanceFilters): Promise<Instance[]>;
+}
+
+// Docker implementation (current)
+export class DockerOrchestrator implements McpOrchestrator {
+  private docker: Docker;
+  // ... existing Docker implementation
+}
+
+// Kubernetes implementation (future-ready)
+export class KubernetesOrchestrator implements McpOrchestrator {
+  private k8sClient: KubernetesClient;
+
+  constructor(config: K8sConfig) {
+    this.k8sClient = new KubernetesClient(config);
+  }
+
+  async startServer(server: McpServer, config: any): Promise<string> {
+    const pod = {
+      apiVersion: 'v1',
+      kind: 'Pod',
+      metadata: {
+        name: `mcp-${server.name}-${Date.now()}`,
+        labels: {
+          'mcpctl.io/server': server.name,
+          'mcpctl.io/managed': 'true'
+        }
+      },
+      spec: {
+        containers: [{
+          name: 'mcp-server',
+          image: server.image || 'node:20-alpine',
+          command: this.buildCommand(server),
+          env: this.buildEnvVars(config),
+          resources: {
+            requests: { memory: '128Mi', cpu: '100m' },
+            limits: { memory: '512Mi', cpu: '500m' }
+          }
+        }],
+        restartPolicy: 'Always'
+      }
+    };
+
+    const created = await this.k8sClient.createPod(pod);
+    return created.metadata.name;
+  }
+
+  // ... other K8s implementations
+}
+
+// Factory based on configuration
+export function createOrchestrator(config: OrchestratorConfig): McpOrchestrator {
+  switch (config.type) {
+    case 'docker': return new DockerOrchestrator(config);
+    case 'kubernetes': return new KubernetesOrchestrator(config);
+    default: throw new Error(`Unknown orchestrator: ${config.type}`);
+  }
+}
+```
+
+Configuration:
+```yaml
+orchestrator:
+  type: docker  # or 'kubernetes'
+  docker:
+    socketPath: /var/run/docker.sock
+  kubernetes:
+    namespace: mcpctl
+    kubeconfig: /path/to/kubeconfig
+```
+
+**Test Strategy:**
+
+Unit test orchestrator interface compliance for both implementations. Integration test Docker implementation. Mock Kubernetes API for K8s implementation tests.
+
+## Subtasks
+
+### 17.1. Define K8s-specific interfaces and write TDD tests for KubernetesOrchestrator
+
+**Status:** pending  
+**Dependencies:** None  
+
+Extend the McpOrchestrator interface (from Task 6) with Kubernetes-specific types and write comprehensive Vitest unit tests for all KubernetesOrchestrator methods BEFORE implementation using mocked @kubernetes/client-node.
+
+**Details:**
+
+Create src/shared/src/types/kubernetes.ts with K8s-specific types:
+
+```typescript
+import { McpOrchestrator, McpServer, InstanceStatus, LogOptions, InstanceFilters } from './orchestrator';
+
+export interface K8sConfig {
+  namespace: string;
+  kubeconfig?: string;  // Path to kubeconfig file
+  inCluster?: boolean;  // Use in-cluster config
+  context?: string;     // Specific kubeconfig context
+}
+
+export interface K8sPodMetadata {
+  name: string;
+  namespace: string;
+  labels: Record<string, string>;
+  annotations: Record<string, string>;
+  uid: string;
+}
+
+export interface K8sResourceRequirements {
+  requests: { memory: string; cpu: string };
+  limits: { memory: string; cpu: string };
+}
+
+export interface K8sSecurityContext {
+  runAsNonRoot: boolean;
+  runAsUser: number;
+  readOnlyRootFilesystem: boolean;
+  allowPrivilegeEscalation: boolean;
+  capabilities: { drop: string[] };
+}
+```
+
+Create src/mcpd/tests/unit/services/kubernetes-orchestrator.test.ts with comprehensive TDD tests:
+
+1. Constructor tests: verify kubeconfig loading (file path vs in-cluster), namespace validation, error handling for missing config
+2. startServer() tests: verify Pod spec generation includes security context, resource limits, labels, command building, env vars
+3. stopServer() tests: verify graceful pod termination, wait for completion, error handling for non-existent pods
+4. getStatus() tests: verify status mapping from K8s pod phases (Pending, Running, Succeeded, Failed, Unknown) to InstanceStatus
+5. getLogs() tests: verify log options (tail, follow, since, timestamps) are mapped correctly to K8s log API
+6. listInstances() tests: verify label selector filtering works, pagination handling for large deployments
+
+Mock @kubernetes/client-node CoreV1Api using vitest.mock() with proper type definitions. All tests should fail initially (TDD red phase).
+
+### 17.2. Implement KubernetesOrchestrator class with Pod security contexts and resource management
+
+**Status:** pending  
+**Dependencies:** 17.1  
+
+Implement the KubernetesOrchestrator class using @kubernetes/client-node, with all methods passing TDD tests from subtask 1, including SRE-approved pod security contexts, resource requests/limits, and proper label conventions.
+
+**Details:**
+
+Install @kubernetes/client-node in src/mcpd. Create src/mcpd/src/services/kubernetes-orchestrator.ts:
+
+```typescript
+import * as k8s from '@kubernetes/client-node';
+import { McpOrchestrator, McpServer, InstanceStatus, LogOptions, InstanceFilters, Instance } from '@mcpctl/shared';
+import { K8sConfig, K8sSecurityContext, K8sResourceRequirements } from '@mcpctl/shared';
+
+export class KubernetesOrchestrator implements McpOrchestrator {
+  private coreApi: k8s.CoreV1Api;
+  private namespace: string;
+
+  constructor(config: K8sConfig) {
+    const kc = new k8s.KubeConfig();
+    if (config.inCluster) {
+      kc.loadFromCluster();
+    } else if (config.kubeconfig) {
+      kc.loadFromFile(config.kubeconfig);
+    } else {
+      kc.loadFromDefault();
+    }
+    if (config.context) kc.setCurrentContext(config.context);
+    this.coreApi = kc.makeApiClient(k8s.CoreV1Api);
+    this.namespace = config.namespace;
+  }
+
+  async startServer(server: McpServer, config: any): Promise<string> {
+    const podName = `mcp-${server.name}-${Date.now()}`;
+    const pod: k8s.V1Pod = {
+      apiVersion: 'v1',
+      kind: 'Pod',
+      metadata: {
+        name: podName,
+        namespace: this.namespace,
+        labels: {
+          'mcpctl.io/server': server.name,
+          'mcpctl.io/managed': 'true',
+          'app.kubernetes.io/name': `mcp-${server.name}`,
+          'app.kubernetes.io/component': 'mcp-server',
+          'app.kubernetes.io/managed-by': 'mcpctl'
+        },
+        annotations: {
+          'mcpctl.io/created-at': new Date().toISOString()
+        }
+      },
+      spec: {
+        containers: [{
+          name: 'mcp-server',
+          image: server.image || 'node:20-alpine',
+          command: this.buildCommand(server),
+          env: this.buildEnvVars(config),
+          resources: this.getResourceRequirements(config),
+          securityContext: this.getSecurityContext()
+        }],
+        securityContext: { runAsNonRoot: true, runAsUser: 1000, fsGroup: 1000 },
+        restartPolicy: 'Always',
+        serviceAccountName: config.serviceAccount || 'default'
+      }
+    };
+    const created = await this.coreApi.createNamespacedPod(this.namespace, pod);
+    return created.body.metadata!.name!;
+  }
+
+  private getSecurityContext(): k8s.V1SecurityContext {
+    return {
+      runAsNonRoot: true,
+      runAsUser: 1000,
+      readOnlyRootFilesystem: true,
+      allowPrivilegeEscalation: false,
+      capabilities: { drop: ['ALL'] }
+    };
+  }
+
+  private getResourceRequirements(config: any): k8s.V1ResourceRequirements {
+    return {
+      requests: { memory: config.memoryRequest || '128Mi', cpu: config.cpuRequest || '100m' },
+      limits: { memory: config.memoryLimit || '512Mi', cpu: config.cpuLimit || '500m' }
+    };
+  }
+  // ... implement stopServer, getStatus, getLogs, listInstances
+}
+```
+
+Implement all remaining methods with proper error handling and K8s API error translation.
+
+### 17.3. Implement createOrchestrator factory function and configuration schema
+
+**Status:** pending  
+**Dependencies:** 17.2  
+
+Create the orchestrator factory function that instantiates DockerOrchestrator or KubernetesOrchestrator based on configuration, with Zod schema validation and configuration file support.
+
+**Details:**
+
+Create src/mcpd/src/services/orchestrator-factory.ts:
+
+```typescript
+import { z } from 'zod';
+import { McpOrchestrator } from '@mcpctl/shared';
+import { DockerOrchestrator } from './container-manager'; // From Task 6
+import { KubernetesOrchestrator } from './kubernetes-orchestrator';
+
+const DockerConfigSchema = z.object({
+  socketPath: z.string().default('/var/run/docker.sock'),
+  host: z.string().optional(),
+  port: z.number().optional(),
+  network: z.string().default('mcpctl-network')
+});
+
+const KubernetesConfigSchema = z.object({
+  namespace: z.string().default('mcpctl'),
+  kubeconfig: z.string().optional(),
+  inCluster: z.boolean().default(false),
+  context: z.string().optional()
+});
+
+const OrchestratorConfigSchema = z.discriminatedUnion('type', [
+  z.object({ type: z.literal('docker'), docker: DockerConfigSchema }),
+  z.object({ type: z.literal('kubernetes'), kubernetes: KubernetesConfigSchema })
+]);
+
+export type OrchestratorConfig = z.infer<typeof OrchestratorConfigSchema>;
+
+export function createOrchestrator(config: OrchestratorConfig): McpOrchestrator {
+  const validated = OrchestratorConfigSchema.parse(config);
+  switch (validated.type) {
+    case 'docker':
+      return new DockerOrchestrator(validated.docker);
+    case 'kubernetes':
+      return new KubernetesOrchestrator(validated.kubernetes);
+    default:
+      throw new Error(`Unknown orchestrator type`);
+  }
+}
+```
+
+Create src/mcpd/src/config/orchestrator.ts for loading config from environment variables and config files (supporting both YAML and JSON). Write TDD tests in src/mcpd/tests/unit/services/orchestrator-factory.test.ts BEFORE implementation:
+
+1. Test factory creates DockerOrchestrator when type='docker'
+2. Test factory creates KubernetesOrchestrator when type='kubernetes'
+3. Test factory throws on invalid type
+4. Test Zod validation rejects invalid configs
+5. Test default values are applied correctly
+6. Test config loading from MCPCTL_ORCHESTRATOR_TYPE env var
+
+### 17.4. Implement K8s NetworkPolicy and PersistentVolumeClaim builders for MCP server isolation
+
+**Status:** pending  
+**Dependencies:** 17.2  
+
+Create resource builders for Kubernetes NetworkPolicy (network isolation between MCP servers) and PersistentVolumeClaim (for stateful data MCPs like caching or GPU providers) with proper annotations for observability.
+
+**Details:**
+
+Create src/mcpd/src/services/k8s-resources.ts with resource builder functions:
+
+```typescript
+import * as k8s from '@kubernetes/client-node';
+
+export interface NetworkPolicyConfig {
+  serverName: string;
+  namespace: string;
+  allowEgress?: string[];  // CIDR blocks or service names to allow
+  allowIngress?: string[]; // Pod labels allowed to connect
+}
+
+export function buildNetworkPolicy(config: NetworkPolicyConfig): k8s.V1NetworkPolicy {
+  return {
+    apiVersion: 'networking.k8s.io/v1',
+    kind: 'NetworkPolicy',
+    metadata: {
+      name: `mcp-${config.serverName}-netpol`,
+      namespace: config.namespace,
+      labels: { 'mcpctl.io/server': config.serverName, 'mcpctl.io/managed': 'true' }
+    },
+    spec: {
+      podSelector: { matchLabels: { 'mcpctl.io/server': config.serverName } },
+      policyTypes: ['Ingress', 'Egress'],
+      ingress: [{
+        from: [{ podSelector: { matchLabels: { 'mcpctl.io/component': 'local-proxy' } } }]
+      }],
+      egress: config.allowEgress?.map(cidr => ({
+        to: [{ ipBlock: { cidr } }]
+      })) || [{ to: [{ ipBlock: { cidr: '0.0.0.0/0' } }] }]  // Default: allow all egress
+    }
+  };
+}
+
+export interface PVCConfig {
+  serverName: string;
+  namespace: string;
+  storageSize: string;  // e.g., '1Gi'
+  storageClass?: string;
+  accessModes?: string[];
+}
+
+export function buildPVC(config: PVCConfig): k8s.V1PersistentVolumeClaim {
+  return {
+    apiVersion: 'v1',
+    kind: 'PersistentVolumeClaim',
+    metadata: {
+      name: `mcp-${config.serverName}-data`,
+      namespace: config.namespace,
+      labels: { 'mcpctl.io/server': config.serverName, 'mcpctl.io/managed': 'true' },
+      annotations: {
+        'mcpctl.io/purpose': 'mcp-server-cache',
+        'mcpctl.io/created-at': new Date().toISOString()
+      }
+    },
+    spec: {
+      accessModes: config.accessModes || ['ReadWriteOnce'],
+      storageClassName: config.storageClass,
+      resources: { requests: { storage: config.storageSize } }
+    }
+  };
+}
+
+export function buildGpuAffinityRules(gpuType: string): k8s.V1Affinity {
+  return {
+    nodeAffinity: {
+      requiredDuringSchedulingIgnoredDuringExecution: {
+        nodeSelectorTerms: [{
+          matchExpressions: [{
+            key: 'nvidia.com/gpu.product',
+            operator: 'In',
+            values: [gpuType]
+          }]
+        }]
+      }
+    }
+  };
+}
+```
+
+Write TDD tests in src/mcpd/tests/unit/services/k8s-resources.test.ts verifying all resource builders generate valid K8s manifests.
+
+### 17.5. Create integration tests with kind/k3d and document K8s deployment architecture
+
+**Status:** pending  
+**Dependencies:** 17.2, 17.3, 17.4  
+
+Build integration test suite using kind or k3d for local K8s cluster testing, create comprehensive SRE documentation covering deployment architecture, resource recommendations, and network requirements.
+
+**Details:**
+
+Create src/mcpd/tests/integration/kubernetes/ directory with integration tests:
+
+1. Create setup script src/mcpd/tests/integration/kubernetes/setup-kind.ts:
+```typescript
+import { execSync } from 'child_process';
+
+export async function setupKindCluster(): Promise<void> {
+  execSync('kind create cluster --name mcpctl-test --config tests/integration/kubernetes/kind-config.yaml', { stdio: 'inherit' });
+}
+
+export async function teardownKindCluster(): Promise<void> {
+  execSync('kind delete cluster --name mcpctl-test', { stdio: 'inherit' });
+}
+```
+
+2. Create kind-config.yaml with proper resource limits
+3. Create kubernetes-orchestrator.integration.test.ts testing:
+   - Pod creation and deletion lifecycle
+   - Status monitoring through pod phases
+   - Log retrieval from running pods
+   - NetworkPolicy enforcement (cannot reach blocked endpoints)
+   - PVC mounting for stateful MCPs
+
+4. Create src/mcpd/docs/KUBERNETES_DEPLOYMENT.md documenting:
+   - Architecture overview: mcpctl namespace, resource types, label conventions
+   - Security: Pod security standards (restricted), NetworkPolicies, ServiceAccounts
+   - SRE recommendations: HPA configurations, PDB templates, monitoring with Prometheus labels
+   - Resource sizing guide: Small (128Mi/100m), Medium (512Mi/500m), Large (2Gi/1000m)
+   - Network requirements: Required egress rules per MCP server type, ingress from local-proxy
+   - Troubleshooting: Common issues, kubectl commands, log access
+   - GPU support: Node affinity, NVIDIA device plugin requirements
+
+5. Create example manifests in src/mcpd/examples/k8s/:
+   - namespace.yaml, rbac.yaml, networkpolicy.yaml, sample-mcp-pod.yaml
+
+Integration tests should skip gracefully when kind is not available (CI compatibility).