Meta Segment Anything Model 3

Original Article ↗ Hacker News Discussion ↗

Model capabilities and significance

  • Many commenters find SAM3 extremely impressive, especially its open-vocabulary, text-prompted segmentation on images and video.
  • Several people describe it as a potential “GPT moment” for computer vision, particularly as a teacher model for distilling smaller, real‑time models.
  • Text as the core interface plus easy integration with LLMs is seen as a major unlock for building higher‑level, multimodal systems.

Applications: prototyping, labeling, and tools

  • Strong interest in rapid prototyping: going from unlabeled video to a fine‑tuned real‑time segmentation model with minimal human effort.
  • Labeling/“autolabel” workflows: some claim SAM3 can automate ~90% of image annotation, flipping data prep to “models with human supervision.”
  • Use cases discussed: video object removal, person de‑identification, background removal, medical imaging, industrial inspection, and game asset generation.

Video, streaming, and editing

  • Built‑in streaming is highlighted as a major improvement over SAM2, which required custom hacks to avoid memory blow‑up on long sequences.
  • Real‑time use is debated: Meta claims ~30 ms per image on high‑end GPUs, but hosted APIs report ~300–400 ms per request; some see it as mainly a distillation teacher rather than a deployable edge model.
  • Video editors (DaVinci Resolve, After Effects plugins, hobby tools) already use related models; SAM3‑level quality is seen as highly desirable for rotoscoping/greenscreen and object removal.

3D reconstruction

  • The SAM3D component impresses people with speed and handling of occlusions; discussion centers on whether it outputs meshes, splats, or both.
  • Demo UX is criticized for making export non‑obvious, but code and weights are available for local use.

Strengths and weaknesses on niche tasks

  • Works well on transparent objects like glass and on children’s drawings for recognition, though some say it traces poorly compared to specialized background‑removal models.
  • Struggles with very fine or abstract structures (e.g., PCB traces, tiny defects, some medical and ultrasound imagery), where classic CV or U‑Net–style models still dominate.

Licensing, ecosystem, and Meta’s role

  • License: custom, commercially usable, with an acceptable‑use policy (e.g., military restrictions) and a requirement to keep the same license on redistribution.
  • Some praise Meta’s pattern of releasing strong open‑weights models and tooling; others argue this is strategic “commoditize your complement” rather than altruism.