Phind-70B: Closing the code quality gap with GPT-4 Turbo while running 4x faster

Original Article ↗ Hacker News Discussion ↗

A new 70B-parameter code-focused language model from Phind is drawing attention for claiming GPT‑4‑level coding quality while running significantly faster on H100 GPUs. Developers report mixed but often positive real‑world results: many find it strong for code generation and technical Q&A, especially in its “chat” mode, while others say GPT‑4 still wins on difficult logic, nuanced API use, and complex CI or retry logic examples. Broader themes include skepticism about benchmark overfitting, questions about hardware, quantization and context windows, requests for an API and better UX, and a sense that specialized coding models may increasingly complement or compete with general‑purpose LLMs.

Model quality vs GPT‑4 / other LLMs

  • Many users find Phind‑70B very fast and strong for coding, sometimes “GPT‑4‑level” for practical dev tasks, especially in code-focused “Chat/Code” mode.
  • Others report clearly worse results than GPT‑4 for:
    • Logical puzzles and trick questions.
    • Nuanced API/doc summarization and IoT/security design questions.
    • Subtle coding tasks (e.g., Go RoundTripper with POST retries, CI pipelines) where GPT‑4 caught more edge cases and best practices.
  • Several note GPT‑4 Turbo is weaker than the original GPT‑4 for code; Phind‑70B may beat Turbo but not GPT‑4 “classic.”
  • DeepSeek, Mistral, Gemini, etc. are referenced; some say DeepSeek Coder is the strongest open model they’ve run locally, but concede Phind‑70B is impressive.

Benchmarks, evaluation, and specialization

  • Multiple commenters distrust current code benchmarks (HumanEval and open‑LLM leaderboards) due to contamination, easy “cheats,” and poor task design.
  • Some argue model size is no longer a simple proxy for quality; others counter that long‑tail coverage still favors very large models like GPT‑4.
  • There's interest in better, more realistic, human‑judged coding benchmarks and arena‑style head‑to‑head evaluations.

Serving, training, and infrastructure

  • Phind says it uses NVIDIA TensorRT‑LLM on H100s; commenters infer Triton is likely involved.
  • Discussion of memory requirements: 70B at 4‑bit fits in ~35–48 GB VRAM; full 16‑bit would be ~140 GB, implying multi‑GPU setups.
  • Team reports a literally “melted” H100 and a relatively high H100 failure rate; some suggest airflow/PLX issues and mention FP8‑related throttling on other GPUs.

Product behavior, UX, and modes

  • Strong praise for:
    • Speed and willingness to emit detailed code instead of refusing.
    • Web search + LLM integration for technical queries; some switch from Google.
  • Weak spots and bugs:
    • 70B selection sometimes silently falls back to 34B, especially when not logged in.
    • Public chat URLs can be edited by anyone, altering visible answers.
    • RAG/search can “pollute” answers; Chat/Code mode often performs better than Search mode.
    • Model occasionally fails to read its own blog page (e.g., context window question).
  • Requests for:
    • An OpenAI‑compatible API, more payment options, mobile apps, better chat history organization, and improved accessibility labels.

Openness, ecosystem, and future

  • Phind has released earlier 34B weights and states intent to release newer 34B and eventually 70B weights.
  • Some criticize it as another closed, walled‑garden model until weights and APIs are broadly available.
  • Broader discussion touches on rapid proliferation of models, the need for better filtering of useful research, and the likely rise of meta‑LLMs that call other models via APIs.