Tabby: Self-hosted AI coding assistant

Original Article ↗ Hacker News Discussion ↗

A self-hosted AI coding assistant called Tabby is drawing interest as an alternative to cloud tools like GitHub Copilot, offering on-premise deployment, team features, and support for Nvidia, AMD, and Apple Silicon hardware. Commenters weigh its practical limits—model size, hardware requirements, context length, and IDE support—against the benefits of keeping code and telemetry in-house, while also probing its open-source licensing, user caps, and data collection. The conversation broadens into concerns about AI-generated “junior-level” code, long-term developer skill erosion, and uneven hiring practices at the company behind the project.

Overview of Tabby and Capabilities

  • Self-hosted AI coding assistant with code completion and “codebase chat,” positioned as an on‑prem / team platform (SSO, access control, auth).
  • Marketed as one of the few fully self-service on-prem options, with adopters saying performance is competitive with hosted tools.
  • Built-in RAG/doc integration so it can be taught unfamiliar API frameworks via documentation ingestion.

Hardware, Models, and Performance

  • Supports Nvidia (CUDA), AMD (via Vulkan), and Apple Silicon; Macs are “OK for individual use” but not ideal for multi‑user servers.
  • Rule-of-thumb: ~1 GB RAM per 1B parameters (less with heavy quantization). Context length also drives memory needs.
  • Tiny models (1–3B) are “dumb” for conversational coding but fine for tab completions; 7–70B open models can surpass GPT‑4o‑mini for coding if hardware permits.
  • Single‑GPU only by default; multi‑GPU use suggested via external backends like vLLM and OpenAI-compatible endpoints.

Deployment, IDE Support, and Alternatives

  • Designed primarily for shared servers but can run on powerful personal machines or in Docker on‑prem.
  • Community notes Eclipse client exists but is not prominently documented; requests for VS2022, Sublime, Zed, MSVC support.
  • Comparisons with other local setups (Ollama + Continue.dev, Twinny) highlight trade‑offs in ease of use, hardware, and licensing.

Telemetry, Licensing, and Business Model

  • Community Edition collects non‑toggleable IDE/extension telemetry, limited to hardware and model metadata per shared struct.
  • Confusion over “open source but up to 5 users” pricing; others clarify that open source does not mean cost-free for all uses and point to the license.

Code Quality, Skill Development, and Determinism

  • Many worry LLMs generate “junior-level” or inefficient code, and that blind acceptance may stall developer growth.
  • Counterpoints:
    • LLMs can accelerate capable devs and serve as a new abstraction layer, similar to moving from assembly to high-level languages.
    • Poor code quality self-corrects through tests, debugging, and maintenance pressures.
  • Long subthread on determinism: traditional compilers vs stochastic LLMs, temperature/seed control, and whether nondeterminism is acceptable for production code.

Critiques of Company Practices

  • One commenter reports an unpaid, multi‑round, take‑home–heavy interview ending in ghosting, sparking broader criticism of such hiring processes as disrespectful and a red flag.