SQL pipe syntax available in public preview in BigQuery

Original Article ↗ Hacker News Discussion ↗

Origins of pipe syntax

  • Several comments trace pipe-style composition back to Unix pipelines in the 1970s.
  • The specific |> function-pipe operator is linked to ML-family languages (Isabelle/ML, F#, OCaml) from the 1990s, later popularized in F# and then picked up by Elixir.
  • People distinguish Unix “bytes in, bytes out” pipes from higher‑level, function-chaining pipes in functional languages.

Comparisons to other pipe / query languages

  • Kusto Query Language (KQL) is repeatedly cited as a strong precedent; some call it “the best query language,” others see it as only an incremental improvement over SQL.
  • PRQL and Malloy are mentioned as cleaner, pipe-oriented languages that compile to SQL; PRQL’s function system and extensibility are highlighted, as is a DuckDB extension.
  • LogsQL, ClickHouse’s KQL mode, and LookML/Malloy are given as further examples of pipe-based or transpiled query syntaxes.
  • Some note that InfluxDB’s Flux (also pipe-based) was dropped due to complexity and low adoption.

Perceived benefits of BigQuery’s pipe SQL

  • Many are enthusiastic: they find the stepwise, linear workflow more natural for data exploration and cleaning, especially for iterative filtering, aggregation, JSON/array handling, and DBT macros.
  • Advantages cited:
    • Unifying WHERE/HAVING/QUALIFY.
    • Reducing nested CTEs and ugly subqueries; easy to comment out or reorder stages.
    • Closer to data-frame/tidyverse / KQL mental models and easier integration with host languages.
    • Table-valued functions plus pipes allow reusable higher-order transformations.

Concerns and drawbacks

  • Some worry this further fragments SQL dialects and increases complexity; they’d prefer higher-level languages that transpile to SQL.
  • Others argue transpilers/ORMs have leaky abstractions and poor optimization compared to engine-native syntax.
  • A few find pipeline style harder to read than modular CTEs because it increases “state in your head,” especially for long chains; best practice suggestions include ending with an explicit SELECT to clarify outputs.
  • There is debate over whether this is essentially “SELECT * in disguise” and whether overuse will harm clarity.
  • Some simply prefer traditional SQL aesthetics and don’t feel a need for pipes.

Standardization and adoption

  • Several commenters hope this syntax spreads to Postgres, MySQL, SQLite, and note early support in Databricks and experiments in SQLite and DuckDB.
  • Others argue the real win would be a standardized, engine-aware pipelined query language used across databases, rather than many slightly different SQL extensions.