Apple Foundation Models

Original Article ↗ Hacker News Discussion ↗

What Apple’s Foundation Models Framework Is

  • Standard Swift API (shipping in iOS/macOS/watchOS/iPadOS/visionOS 27) for language models.
  • Supports both:
    • Apple’s on-device small model (SystemLanguageModel).
    • Cloud models plugged in via a common LanguageModelSession / LanguageModel protocol (e.g., Claude, Gemini).
  • Same API handles streaming, tool calling, structured output; apps can swap models by changing a single constructor.

Claude Integration Details

  • Claude is exposed as a Swift package implementing Apple’s LanguageModel protocol.
  • Requests from the device go directly to api.anthropic.com; Apple is not in the network path and doesn’t see prompts or responses.
  • Claude via this framework is cloud-only; offline, only Apple’s on-device ~3B-parameter model keeps working.
  • Usage is billed to a regular Anthropic account at standard API pricing.

On-Device vs Cloud and Storage Concerns

  • Apple’s current on-device model size is limited (around 3B params); Claude is much larger and remains server-side.
  • Some want shared, system-level caching of third-party local models to avoid each app downloading its own copy; others note Apple only deduplicates system models today.
  • Debate on whether a single, optimized default on-device model is better for UX and resources than many app-specific models.

Business Model, Lock-In, and UX Control

  • Many see this as Apple commoditizing LLM providers while keeping control over the overall UX and branding (“Apple Intelligence” as the visible layer).
  • Speculation about future “Intelligence Store”–style revenue shares or 30% cuts on AI-powered app subscriptions, even if API billing bypasses Apple directly.
  • Some worry Apple could later favor or swap providers without users noticing, reinforcing platform lock-in.

Developer Experience & API Keys

  • For development, the SDK can use a direct API key; for production, docs strongly recommend proxying via the developer’s backend, which injects credentials.
  • This avoids shipping keys in apps but raises concerns about proxy-side access to user prompts and data.
  • Some commenters dislike token-based, variable-cost UX for end users; subscriptions or local models are seen as more user-friendly.

Model Commoditization and Competition

  • Thread debates whether frontier models are becoming interchangeable “commodities” versus still having substantial qualitative differences.
  • Mixed views on which labs are ahead, but broad agreement that:
    • Most app use cases don’t need absolute frontier performance.
    • Unified OS-level APIs will push many apps toward “good-enough” default models, with cloud frontiers reserved for heavier tasks.