Apple Foundation Models

Original Article ↗ Hacker News Discussion ↗

Apple’s new Foundation Models framework provides a unified Swift API that lets iOS and macOS apps swap seamlessly between Apple’s small on‑device models and cloud LLMs like Anthropic’s Claude or Google’s Gemini. Commenters see this both as a developer convenience and a strategic move: it abstracts away individual providers, helps Apple keep UX and branding control over AI features, and could further lock developers and users into the ecosystem. Key concerns center on business models, API key handling via proxies, privacy implications, and how on‑device models and storage will scale as more apps depend on shared AI capabilities.

What Apple’s Foundation Models Framework Is

  • Standard Swift API (shipping in iOS/macOS/watchOS/iPadOS/visionOS 27) for language models.
  • Supports both:
    • Apple’s on-device small model (SystemLanguageModel).
    • Cloud models plugged in via a common LanguageModelSession / LanguageModel protocol (e.g., Claude, Gemini).
  • Same API handles streaming, tool calling, structured output; apps can swap models by changing a single constructor.

Claude Integration Details

  • Claude is exposed as a Swift package implementing Apple’s LanguageModel protocol.
  • Requests from the device go directly to api.anthropic.com; Apple is not in the network path and doesn’t see prompts or responses.
  • Claude via this framework is cloud-only; offline, only Apple’s on-device ~3B-parameter model keeps working.
  • Usage is billed to a regular Anthropic account at standard API pricing.

On-Device vs Cloud and Storage Concerns

  • Apple’s current on-device model size is limited (around 3B params); Claude is much larger and remains server-side.
  • Some want shared, system-level caching of third-party local models to avoid each app downloading its own copy; others note Apple only deduplicates system models today.
  • Debate on whether a single, optimized default on-device model is better for UX and resources than many app-specific models.

Business Model, Lock-In, and UX Control

  • Many see this as Apple commoditizing LLM providers while keeping control over the overall UX and branding (“Apple Intelligence” as the visible layer).
  • Speculation about future “Intelligence Store”–style revenue shares or 30% cuts on AI-powered app subscriptions, even if API billing bypasses Apple directly.
  • Some worry Apple could later favor or swap providers without users noticing, reinforcing platform lock-in.

Developer Experience & API Keys

  • For development, the SDK can use a direct API key; for production, docs strongly recommend proxying via the developer’s backend, which injects credentials.
  • This avoids shipping keys in apps but raises concerns about proxy-side access to user prompts and data.
  • Some commenters dislike token-based, variable-cost UX for end users; subscriptions or local models are seen as more user-friendly.

Model Commoditization and Competition

  • Thread debates whether frontier models are becoming interchangeable “commodities” versus still having substantial qualitative differences.
  • Mixed views on which labs are ahead, but broad agreement that:
    • Most app use cases don’t need absolute frontier performance.
    • Unified OS-level APIs will push many apps toward “good-enough” default models, with cloud frontiers reserved for heavier tasks.