Ollama is now available on Windows in preview

Original Article ↗ Hacker News Discussion ↗

Ollama’s new Windows preview is prompting renewed interest in running large language models locally, with many users praising how easily it turns consumer PCs—especially gaming rigs—into practical AI workstations. Much of the debate centers on fragmented GPU support (NVIDIA vs. AMD, ROCm vs. Vulkan/OpenCL), architectural choices like exposing models via local HTTP servers, and how front-ends such as Open-WebUI, LM Studio, and Msty integrate with Ollama. Commenters also compare model quality for non-English languages, describe real-world uses from code autocomplete to writing assistance, and highlight the potential to cut cloud AI costs by offloading “easy” prompts to local models.

Platform availability and Windows preview

  • Ollama was already popular on macOS and Linux; native Windows support removes the need for WSL, which some found powerful but inconvenient.
  • Users appreciate that it now runs similarly well on Windows and macOS, including GPU support where available.

GPU and driver support (especially AMD)

  • Repeated frustration over lack of polished AMD GPU support, especially on Windows; many regret buying AMD for AI workloads.
  • Some report ROCm working well on recent AMD cards under Linux; others describe kernel panics, dropped support for consumer GPUs, and painful installs.
  • Alternatives mentioned: OpenCL and Vulkan backends via llama.cpp; OpenCL often “works” but with mixed performance vs ROCm.
  • Maintainers say AMD GPU support is on the roadmap and is already being worked on with high-end Radeon hardware; Windows ROCm support is tracked separately.
  • NVIDIA is seen as more reliable for AI overall, though some Linux users dislike its driver situation on fast-moving distros.

Frontends, tooling, and editor integration

  • Open-WebUI (formerly ollama-webui) is highlighted as a fast-evolving chat interface with conversation search.
  • Another desktop app (Msty) provides a unified UI for local and online models, now on Windows, with Linux planned and AMD support under exploration.
  • VS Code integrations: Continue.dev and Cody can use Ollama for tab-autocomplete and (soon) chat/refactoring.

Language capabilities and model choice

  • Llama 2 (especially 13B chat) performs poorly in Chinese and Japanese; mixing languages and hallucinating about its own abilities.
  • Others recommend Chinese-focused or multilingual models (e.g., Qwen, Yi) from the Ollama library, or raw/base variants that may preserve non-English ability better.

Architecture: server vs in-process

  • Debate over why local tools run as HTTP servers instead of embedding models directly into UIs.
  • Pro-server arguments: avoid repeated model loading, share one model across multiple clients and machines, easier packaging (containers), OpenAI-compatible APIs, and remote/headless use.
  • Critics dislike always-on local services and unauthenticated listeners but are often using strong machines that could embed models directly.

Stability, performance, and installation issues

  • Reports of kernel panics on macOS when using near-maximum memory and with AMD ROCm on Linux.
  • Some GPUs (e.g., NVIDIA Quadro P3000) are not detected; maintainers state support should exist and ask for logs.
  • Windows installers and AI apps frequently trigger antivirus false positives; code signing and installer tooling are suggested mitigations.

Alternative tools and comparisons

  • LM Studio is praised for simple setup and one-click model downloads, but criticized for opaque cache usage, weak search, and restrictive model import.
  • Users value Ollama’s simplicity, open-source nature, and ability to offload “easy” prompts locally while reserving cloud models for harder tasks.