Perplexity Labs Playground

Original Article ↗ Hacker News Discussion ↗

Perplexity Labs’ new web playground lets users try a range of open‑source large language models, impressing many with its speed, minimalist interface, and ability to run powerful models that are hard to host locally. However, users report inconsistent quality: smaller or non‑instruct models often hallucinate, mishandle simple queries like “what is this?” or basic math, and can be easily steered into misinformation or unethical content despite nominal safety rules. There’s also confusion over whether it’s a chatbot or a search‑augmented answer engine, alongside criticism of UX choices, censorship behavior, and opaque behind‑the‑scenes infrastructure.

Overall Impressions & Performance

  • Many praise the Playground as fast, responsive, minimal, and convenient for trying multiple open-source LLMs without setup.
  • Others find quality inconsistent: speed is impressive, but answers often wrong, shallow, or weirdly off-topic.
  • Some feel it is not better than ChatGPT 3.5; others find specific models (e.g., Mixtral, CodeLlama) strong.

Model Behavior & Answer Quality

  • The default pplx-7b-online model often behaves oddly: misidentifies itself (Google Lens, Alexa, IBM Watson, random people), and treats generic prompts as web searches.
  • Instruct-tuned models (e.g., Mixtral-instruct, pplx-70b-chat, Mistral-7B-instruct) generally give much better, more coherent answers.
  • Users see clear failures on basic arithmetic, physics, specific debugging questions, and multilingual queries; some models stubbornly defend wrong answers.

Search Integration & Bias

  • Online models heavily incorporate web search/RAG. For very generic prompts like “what is this” or “what’s your name,” they parrot top Google results (e.g., Google Lens, Eminem).
  • This leads to strong apparent bias toward whatever ranks highest in search, including niche blog posts and riddles sites.

Safety, Guardrails & Misuse

  • Guardrails appear uneven: some models refuse porn but freely describe harmful content (e.g., ethnic extermination, dangerous instructions) or help argue flat-earth or racist positions when framed as “thought experiments.”
  • Some users like the lack of “nannying”; others see this as a serious misinformation and ethics risk.

UX & Product Positioning

  • Confusion over what the Playground is: some expect a straightforward chatbot; in practice the default model behaves more like a search-driven Q&A engine.
  • No-signup usage and easy model switching get praise; others call the initial experience a UX fail due to lack of explanation and weak default model choice.

Local vs Hosted Models & Infrastructure

  • Several compare Playground models to running Mistral/Llama locally via ollama/llama.cpp, discussing quantization, VRAM, tokens/sec, and hardware differences.
  • There is appreciation for the hidden engineering and cost optimization needed to serve large models at scale, with curiosity about Perplexity’s infrastructure and custom search index.