My AI costs went from $100 to less than $1/day: Fine-tuning Mixtral with GPT4
An AI founder describes cutting daily costs from around $100 to under $1 by using GPT‑4 to generate training data, then fine-tuning an open model (Mixtral) to replace OpenAI’s API in production. Commenters debate whether this violates OpenAI’s terms of service and what legal or practical risks small companies actually face, especially given OpenAI’s own controversial data sourcing. Others question the technical and ethical trade‑offs, noting that distilled models often underperform GPT‑4, may be user‑hostile if swapped in quietly, and that for narrow tasks it might be cheaper to create high‑quality training data by hand.
OpenAI Terms of Service & Legality
- Multiple commenters argue the described workflow (using GPT‑4 output to fine‑tune Mixtral) is explicitly against OpenAI’s ToS, which prohibit using output to develop competing AI models, except for specific internal use cases.
- Others question whether a niche, application-specific model really “competes” with OpenAI’s products, suggesting the clause may target generic LLM APIs and ChatGPT-style services.
- Some note that academic or non-productized work is likely safer, but once commercialized it becomes more vulnerable to ToS action.
Enforcement Risk & Legal Uncertainty
- Many believe OpenAI’s main realistic remedy is cutting off API access rather than suing, especially for small users.
- Several point out that court challenges (e.g., ongoing media lawsuits) may eventually decide if training or output restrictions are enforceable at all.
- Others stress that even if one feels morally justified, being the test case in court or losing API access could ruin a business or complicate acquisition prospects.
Ethics, Fairness, and “Piracy”
- A strong theme: OpenAI allegedly trained on content against other platforms’ ToS and without consent, so some see “stealing from a thief” as morally acceptable.
- Others push back, distinguishing “hacker spirit” from unethical behavior and emphasizing contractual obligations, even if one dislikes them.
- There is tension between people who prioritize information freedom and those who prioritize legal/contractual prudence.
Technical Approach & Model Quality
- Some praise the method as a smart, cheap way to approximate GPT‑4 behavior for a narrow domain.
- Skeptics argue fine‑tuned smaller models will still be noticeably weaker under stress, and that for limited domains one could simply hand‑craft or hand‑label data.
- There’s discussion that synthetic data from LLMs is overused and manual labeled data can sometimes be faster, cheaper, and more consistent.
Business Models, Costs, and “Bait‑and‑Switch”
- Commenters note many companies already distill GPT‑4 into open models, brand them as proprietary, and claim AI leadership.
- Some see the pattern as: launch on GPT‑4, then silently swap to a cheaper in-house model once users are hooked, calling this user-hostile “enshittification.”
- Others discuss the economics: OpenAI pricing may rise; self-hosted or open models might become necessary for margins, but fine-tuning and infra aren’t free and can require expertise.
App-Specific Concerns
- The showcased app is criticized for lacking ToS, privacy policy, and pricing transparency while collecting detailed personal data, which some label “shady.”
- The maker acknowledges these issues and agrees they need clearer policies and transparency.