Show HN: I made a website to semantically search ArXiv papers
A new open-source tool offers semantic search over arXiv, aiming to surface conceptually related papers rather than relying on exact keyword matches. Commenters probe its technical choices (MixedBread embeddings, binarization, Milvus) and trade-offs between speed and recall, while contrasting semantic retrieval with traditional engines like Google Scholar and arXiv’s own keyword search. Many see potential for literature reviews, domain-specific workflows, and local or extended versions (bioRxiv/medRxiv, patents, internal corpora), but stress the need for features such as date filters, citation/review metadata, “similar papers” navigation, and LLM-powered summarization.
Project scope and related tools
- Site provides semantic search over arXiv, with companion versions for bioRxiv and medRxiv (not yet fully synchronized).
- Compared to tools like Semantic Scholar, arxivxplorer, OpenAlex-based systems, Research Rabbit, emergentmind, and academic search / workflow tools (undermind, scite, elicit, paper-qa, txtai, paperai, paperetl, Semantra).
- Some suggest integrating with external tools (e.g., paper-qa, OpenReview) and drawing on existing arXiv embeddings datasets.
Technical implementation and performance
- Uses MixedBread embeddings, chosen for small size, strong leaderboard performance, binary and matryoshka support.
- Embeddings are binarized and stored in Milvus; binary Hamming search yields large latency improvements (~hundreds of ms).
- Acknowledged tradeoff: top ~10 results are similar to full-precision search, but quality drops quickly beyond that.
- Suggestions include reranking a larger Hamming-retrieved candidate set with full-precision scores, using shorter embeddings instead of binarization, trying brute-force CPU search with SIMD, and exploring hybrid (keyword + vector) retrieval.
- Weekly metadata updates are automated via a Hugging Face Space.
Search quality: strengths and weaknesses
- Semantic search surfaces conceptually similar work even without exact keyword overlap; several users report discovering new relevant papers.
- Others find it weak for niche or overloaded terms (“leaky relu,” “wave function collapse algorithm,” specific astronomy sub-terms), where keyword-based arXiv/Scholar works better.
- Some recommend domain-specific or fine-tuned models to improve technical term handling.
- Author-name search is noted as a poor fit for pure semantic search.
Feature requests and UX issues
- Strong demand for filters, especially date/recency sorting and more dense results (collapsible abstracts).
- Users ask for “similar papers” links, citation/review integration, and explanatory “how to use” guidance (e.g., best to paste abstracts or arXiv IDs).
- Encoding and LaTeX/Markdown rendering bugs are reported.
- Cloudflare bot challenges are a dealbreaker for some, sparking debate about alternatives and broader web centralization problems.
Use cases, limitations, and research workflows
- Seen as useful for exploratory discovery and recommendations, but multiple commenters insist systematic reviews should not rely on semantic search or preprints alone.
- Proposed workflows include literature reviews, technology watch for industry and tax credit work, internal document and code search, and local/offline search via Docker.
- Some envision generative “literature overview” summaries over sets of retrieved papers as a next step.