Revolut moves to self-hosted GLM
A major fintech has shifted much of its AI workload from OpenAI and Anthropic to a self‑hosted GLM model running on rented Nvidia hardware, reportedly cutting per‑token costs by about 90% while lifting internal token usage caps. Commenters debate whether the quality gap between open‑weight and “frontier” models still matters for most coding and back‑office tasks, given regulatory pressures, data‑sovereignty concerns, and the economics of large GPU clusters versus API pricing. The thread also highlights a broader trend: enterprises experimenting with on‑prem or dedicated-cloud LLMs, even as questions linger over hardware depreciation, vendor moats, and how ubiquitous AI will reshape software engineering jobs.
Cost & Infrastructure Economics
- Revolut reports ~$0.30 per million tokens on self‑hosted GLM (via rented 56× B200s), ~1/10 of prior Anthropic/OpenAI spend.
- Rough community estimates: 56 B200s might cost $2.5–3M to buy, or $250–300k/month to rent at public cloud GPU prices, suggesting the switch is worthwhile at high usage.
- Debate over buy vs rent: many say renting metal (from cloud or specialized providers) is more realistic for most; hardware is hard to procure and rapidly depreciates with each Nvidia generation.
- Some expect frontier labs to respond with aggressive enterprise plans (e.g., flat per‑engineer pricing) to preserve margins and relevance.
Model Quality: Open vs Frontier
- Many claim GLM 5.3 and similar open‑weight models are “good enough” for most coding/SE tasks, especially with good prompts and harnesses.
- Others insist frontier models (Claude, GPT, etc.) still provide clearly superior reasoning, autonomy, and code navigation, especially on complex repos; switching back to them feels like a “night and day” upgrade.
- Consensus: quality gap is real but shrinking; for many business workloads, price and control now matter more than absolute peak quality.
Data Privacy, Compliance & “Zero Data Retention”
- For a bank, keeping workflows within controlled infrastructure (or at least a trusted GPU provider) is seen as crucial for regulatory and data‑sovereignty reasons.
- Several argue that “zero data retention” promises from SaaS LLM vendors are just contractual “pinky promises” with legal carve‑outs and no verifiability.
- Suggested hierarchy: own hardware > confidential‑computing providers > generic API vendors. Others counter that audits and contract law are sufficient for many organizations.
Self‑Hosting Experiences & Tooling
- Multiple commenters report success with on‑prem or local open models (Qwen, Gemma, GLM, etc.), often on modest hardware, eliminating data‑trust concerns.
- Emphasis on the harness: logging, tools, standardized APIs, and lessons files matter as much as the base model. Some share setups that mine prior conversations into “lessons” injected into future contexts.
- A few note engine‑level bugs (e.g., vLLM + GLM correctness issues) and wonder about Revolut’s specific stack.
Developer Productivity & Token Rationing
- A key reported gain: removing token rationing. Once tokens were effectively “flat‑rate,” usage exploded and productivity (and exploration) rose.
- Contrasting view: at some companies with unlimited tokens, this led to “illusion of productivity” and lots of AI‑generated slop.
- Several describe that open models follow instructions better when prompts are precise, and that ambiguity (“fix all the problems”) produces worse outcomes.
Labour Market & Skills
- Strong concern that junior dev work is now largely automatable; some teams keep juniors mostly for learning, with seniors reviewing AI‑assisted code.
- Seniors are widely perceived as the primary beneficiaries of AI, becoming far more productive; juniors face hiring headwinds and are urged to learn AI workflows proactively.
- Debate over whether employees should invest personal time/money in AI tools vs employers providing subscriptions, and over broader questions of job security, work–life balance, and societal impact.