Fly Kubernetes

Original Article ↗ Hacker News Discussion ↗

Fly.io’s new “Fly Kubernetes” feature exposes a Kubernetes-compatible API on top of its existing Machines platform, using k3s and Virtual Kubelet to map pods directly to Fly VMs. Commenters weigh the benefits of this approach—declarative configs, tooling compatibility, and pod-per-VM autoscaling—against its limitations and nonstandard behavior, such as incomplete support for multi-container pods and persistent volumes. The move is seen by some as a pragmatic adapter for organizations already invested in Kubernetes, and by others as a risky step away from Fly’s original simplicity and a reminder that platform stability and documentation still need work.

What Fly Kubernetes (FKS) Is

  • Positioned as an optional Kubernetes-compatible interface on top of Fly Machines, not a shift of Fly’s own infrastructure to Kubernetes.
  • Implemented as k3s + Virtual Kubelet running on a single Fly Machine; Pods are mapped 1:1 to Machines.
  • Goal: let teams reuse Kubernetes tooling and declarative YAML on Fly, while delegating scheduling and infrastructure to Fly’s platform.

Positioning vs Other Cloud Offerings

  • Some see FKS as similar in spirit to GKE Autopilot or EKS Fargate: pod-level billing without node management.
  • Others argue it’s mainly a “Kubernetes API shim” to the Fly platform, not a full Kubernetes distribution.
  • Several commenters suggest documentation and marketing should more clearly spell out what FKS is and isn’t, and which Kubernetes features are supported.

Technical Design & Limitations

  • Control plane is single-node k3s with SQLite; not obviously HA. If that Machine dies, API access may be interrupted.
  • Multiple comments question support for multi-container pods and sidecars; current Fly Machines are single-image per Machine, so full parity is unclear and acknowledged as a work in progress.
  • Affinity/anti-affinity semantics change when “node” is a virtual-kubelet with pod-per-VM; some resilience patterns may not translate directly.
  • Persistent storage remains tied to physical hosts via Fly Volumes; Fly can migrate volumes, but hardware failure can still cause data loss, so higher-level DB replication or external databases are recommended.

Kubernetes Ecosystem vs Custom Platforms

  • One view: using Kubernetes taps into an ecosystem of standards, tools, and talent and avoids reinventing orchestration.
  • Counterview: Kubernetes is complex, sometimes overkill; Fly’s custom scheduler and network stack may be simpler for their problem. FKS is just an adapter for users who insist on K8s.

User Reactions & Concerns

  • Some are excited to gain declarative configs and kubectl-based workflows on Fly.
  • Others worry this dilutes Fly’s PaaS simplicity or is a “footgun” that embraces Kubernetes complexity.
  • Reliability concerns are raised about certain Fly regions and about Fly’s overall stability and documentation consistency.
  • Several ask for clearer guarantees on uptime, pod lifetimes, storage behavior, and cross-region behavior; many details remain unclear.