Magika: AI powered fast and efficient file type identification

Original Article ↗ Hacker News Discussion ↗

Google’s Magika project applies a compact neural network to identify file types from content, aiming to outperform traditional signature‑based tools like `file`/libmagic, especially for ambiguous or semi‑structured text formats. Commenters welcome new options for handling messy real‑world uploads and large file corpora, but note that Magika currently supports far fewer formats, is often tens of times slower, and can confidently mislabel unknown or adversarial binaries instead of saying “unknown.” Many see it as a promising research and niche tool or a second‑pass classifier rather than a drop‑in replacement for existing deterministic methods, and question how useful it can be without open training data or robust handling of edge cases like polyglot files.

Scope and Supported Types

  • Model currently recognizes ~116 content types, focused on “major” and common formats.
  • Many commenters note missing but common categories: raw camera images, CAD files, MIDI/music formats, trackers (.mod), several legacy languages (Pascal, COBOL, assembly, APL), DXF, etc.
  • It’s seen as particularly promising for distinguishing similar text-based formats (e.g., CSV vs generic text, markdown, language-specific code).

Accuracy, Failure Modes, and Adversarial Concerns

  • Blog claims >99% accuracy on its dataset, but many commenters report misclassifications: DXF as PowerShell/plain text, HTML as VBA/ASP/generic text, fonts as FLAC/ISO, ROMs as SWF, Gradle Kotlin as Scala, unknown binaries as ZIP, etc.
  • Key criticism: unknown or unsupported formats are often confidently mapped to a known type instead of “unknown”/“data.”
  • This behavior is seen as dangerous in adversarial or security-sensitive contexts (malware detection, AV, upload validation).
  • Some small-scale tests show aggregate accuracy comparable to or slightly better than file, but with very different and less predictable failure modes.

Performance and Resource Use

  • Authors report Magika is ~10× slower than file for single files, ~2× slower in batch.
  • Independent benchmarks on real systems see 30–50× slower runtimes and much higher CPU usage, especially via the CLI and Node/WASM, where model loading dominates.
  • Debate over whether extra energy and latency matter: some say negligible in server workflows; others highlight battery and interactive UX concerns.

Comparison with Existing Tools

  • file/libmagic supports ~1600 types, is very fast, and often returns “data” when unsure.
  • Some argue libmagic is “more capable, predictable, and energy‑efficient,” especially on binaries and fonts.
  • Others note libmagic’s limitations for semi-structured text, zip-based Office formats, and large-scale, messy datasets, where it misclassifies or is too generic.
  • Apache Tika, TrID, ExifTool, and GitHub’s Linguist are mentioned as existing alternatives.

Use Cases and Integration Strategies

  • Internal Google use: routing Gmail/Drive files for antivirus scanning and policy enforcement.
  • External ideas: upload validation (when users lie with extensions), code editors’ language detection, file managers and thumbnails, data recovery (Photorec outputs), secret scanners, web crawls.
  • Several suggest a hybrid approach: use file/libmagic first, then fall back to Magika when results are “data” or low-confidence.

Openness, Extensibility, and AI Skepticism

  • Code and ONNX model are released under Apache, but training code and dataset are not; some say this makes it only partially “open.”
  • Without training pipeline and data, community extension to new formats is unclear.
  • There’s broader skepticism about using ML for a task that is often deterministic via headers and specs, and concern that “AI” branding overpromises compared to a slower, less transparent classifier.