K8s cluster migrations

Kubernetes cluster workload migrations

This project focused on assisting multiple clients' teams in migrating their workloads from legacy clusters to new ClusterAPI managed clusters.

Highlights

• successfully migrated workloads for various clients to new Kubernetes clusters

Details

Analytics & AI

Deployed applications included:

• Apache Flink
• OpenTelemetry (collector, operator)
• Tableau
• custom AI/LLM workloads for model training and in-house AI/LLM application development

Operations

Deployed applications included:

• Grafana
• Thanos
• Teleport
• Atlantis
• kubecost
• Jaeger
• karma
• Strimzi
• Pyroscope
• Tempo
• various in-house tools

In-house Software

Migrations also involved bespoke in-house developed microservice stacks, which comprised the backends and frontends of company's financial and cryptocurrency-related services. These services were built using:

• Rust
• Go
• Java
• TypeScript

Optimizations during the migration process included:

• reducing image sizes to enhance deployment speed
• improving security by avoiding execution as root and eliminating unnecessary runtime dependencies

The migration process often involved:

• automating deployment processes
• adopting GitOps while maintaining release workflows
• customizing application code to integrate with new platform services, such as using Vault CSI and AWS IRSA to avoid static secrets
• building or adjusting CI/CD pipelines, see here for details