Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

Original Article ↗ Hacker News Discussion ↗

AMD’s Lemonade project is presented as a unified, open‑source server for running local generative AI (LLMs, image generation, TTS/STT) with a strong focus on AMD GPUs and Ryzen AI NPUs. Commenters compare it to tools like Ollama and LM Studio, noting that it builds on llama.cpp, offers OpenAI/Ollama/Anthropic‑compatible endpoints, and significantly simplifies the messy setup of ROCm, Vulkan, and multiple runtimes. Enthusiasm centers on its “one-stop” orchestration and AMD optimizations, while skepticism focuses on ROCm stability, the limited real‑world value of NPUs for large models, and proprietary NPU kernels used by FastFlowLM.

What Lemonade Is Trying to Be

  • Positioned as a unified local AI server and management layer focused on AMD hardware.
  • Bundles multiple runtimes/backends: llama.cpp for text/vision, diffusion for images, Whisper-style STT, TTS, and NPU runtimes (FastFlowLM).
  • Exposes OpenAI-, Ollama-, and Anthropic-compatible endpoints so existing tools and UIs can talk to it.
  • Includes its own web UI for model management, configuration, and interaction.

Comparison to Ollama, LM Studio, vLLM

  • Multiple commenters see it as “between Ollama and LM Studio”: more orchestration and multi‑modal support than simple model serving.
  • Under the hood, both Lemonade and Ollama rely on llama.cpp; Lemonade adds AMD-tuned builds and multi-backend routing.
  • A small benchmark on an M1 Max showed Lemonade modestly faster than Ollama for one Qwen3.5 9B prompt, but this is anecdotal.
  • Some prefer using Lemonade’s ROCm‑optimized llama.cpp builds directly instead of the full server.

Performance, ROCm vs Vulkan

  • Reports that Vulkan can outperform ROCm on some AMD GPUs, especially integrated/APUs; others see ROCm faster on high-end cards like 7900 XTX.
  • A linked ROCm issue notes current regressions; expectation is ROCm should be faster if fixed.
  • Users report strong performance on Strix Halo and various Radeon cards, especially with Vulkan and newer kernels.

NPU Role and Limitations

  • NPU support uses FastFlowLM; its NPU kernels are proprietary (free for non‑commercial use, commercial license otherwise).
  • Consensus: NPUs are best for small, always‑on, low‑power models (e.g., STT/TTS, small LLMs, prefill offload), not large chatbot workloads.
  • On Strix Halo, NPU performance is described as underwhelming compared to the GPU/APU but effectively “free” power-wise.

Packaging and Platform Support

  • Provides deb/rpm, Ubuntu PPA, Snap, macOS beta, and container options (though some think Docker instructions should be more prominent).
  • macOS uses Metal now; MLX support is on the roadmap.

Enthusiasm vs Skepticism

  • Enthusiastic AMD users describe Lemonade as the easiest turnkey way to run local AI on AMD (especially Strix Halo).
  • Others criticize ROCm as unstable, complain about crashes when exceeding VRAM, or dismiss Lemonade as unnecessary “slop” over plain llama.cpp with Vulkan.
  • Some worry about vendor-specific stacks and proprietary NPU pieces limiting openness and portability.