omnisearch

Unified web and social media search across 5 providers (Brave, Tavily, Xpoz, Exa, Firecrawl). Auto-routes queries to the best API, supports parallel multi-provider search with deduplication, AI-genera

SKILL.md

GitHub

Omnisearch Skill

Unified search across 5 providers — Brave (Google SERP), Tavily (agent-native answers), Xpoz (social media), Exa (neural/semantic), and Firecrawl (scraping). Auto-classifies queries and routes to the best provider, or runs parallel searches across all with deduplication.

Prerequisites: At least one API key set. See API Key Setup.

Coverage Map

Surface Provider Strength Cost
Google SERP Brave Web + news, freshness filtering, LLM Context API $5/1k
Agent answers Tavily AI-synthesized answers with citations, content extraction Free 1k/mo
Twitter/X Xpoz Tweet search, user timelines, threads, hashtags Free 100k/mo
Reddit Xpoz Post search, subreddit browsing, trending Free 100k/mo
Instagram Xpoz Post search, user profiles Free 100k/mo
Neural/semantic Exa (cross-skill) Research papers, companies, similar pages Per-query
Web scraping Firecrawl (cross-skill) Search + scrape, site crawling Per-page

Token-Efficient Multi-Search

Same principle as Exa's dynamic filtering — search cheaply, filter, then extract selectively.

DO:

# Step 1: Multi-provider search, titles/URLs only
python3 ~/.claude/skills/omnisearch/scripts/omnisearch.py "query" --parallel --no-text

# Step 2: Evaluate titles, pick best URLs

# Step 3: Extract selected URLs
python3 ~/.claude/skills/exa-search/scripts/exa_contents.py URL1 URL2 --highlights --max-chars 3000
# OR
python3 ~/.claude/skills/firecrawl/scripts/firecrawl_api.py scrape URL --only-main-content

DON'T: Run --parallel with full text for 40+ results, then reason over all of it.

Available Scripts

1. omnisearch.py — Meta-Router (start here)

Auto-classifies queries and dispatches to the best provider. Supports forced provider, parallel search, and social mode.

python3 ~/.claude/skills/omnisearch/scripts/omnisearch.py "query" [options]
Quick Example Purpose
... omnisearch.py "Iran oil sanctions" Auto-route to best provider
... omnisearch.py "query" --provider brave Force specific provider
... omnisearch.py "query" --parallel All web providers, merged + deduped
... omnisearch.py "query" --social Twitter search via Xpoz
... omnisearch.py "query" --social --platform reddit Reddit search
... omnisearch.py "query" --answer AI-generated answer (Tavily/Brave)
... omnisearch.py "query" --news --parallel News from all providers
... omnisearch.py "query" --academic Route to Exa academic index
... omnisearch.py "query" --verbose Show routing decisions on stderr

Query Classification (auto-route):

Signal Category Primary Provider
@mentions, #hashtags, "tweet", "reddit", r/xxx Social Media Xpoz
"latest", "breaking", "today", date words News Brave
"paper", "arxiv", "study", "journal" Academic Exa
"who is", "what is", short questions Factual Quick Tavily (with answer)
"company", "startup", "funding", "IPO" Company Research Exa
"documentation", "API", "tutorial", "SDK" Technical Docs Exa
(default) General Web Brave

2. brave_search.py — Brave Web & News Search

Google-indexed SERP with freshness filtering, country targeting, and LLM Context API.

python3 ~/.claude/skills/omnisearch/scripts/brave_search.py "query" [options]
Quick Example Purpose
... brave_search.py "Iran infrastructure" Web search
... brave_search.py "query" --news Dedicated news endpoint
... brave_search.py "query" --summarize LLM Context API (AI summary)
... brave_search.py "query" --freshness pd Past day only
... brave_search.py "query" --freshness pw --extra-snippets Past week + rich context
... brave_search.py "query" --country US --lang en Localized results
... brave_search.py "query" --filter web,news --markdown Multi-type + markdown

Freshness values: pd (past day), pw (past week), pm (past month), py (past year), or YYYY-MM-DDtoYYYY-MM-DD

3. tavily_search.py — Agent-Native Search with Answers

AI-optimized search with grounded answers, topic modes, and content extraction.

python3 ~/.claude/skills/omnisearch/scripts/tavily_search.py "query" [options]
Quick Example Purpose
... tavily_search.py "oil prices" --answer Search + AI answer
... tavily_search.py "query" --advanced --answer Deep search + answer
... tavily_search.py "query" --news --days 3 News, last 3 days
... tavily_search.py "query" --finance Finance topic mode
... tavily_search.py "query" --answer-only Pipe-friendly answer
... tavily_search.py "query" --domains bellingcat.com Domain-restricted
... tavily_search.py extract URL [URL...] Content extraction

Search depths: basic (fast, ~1s) or --advanced (comprehensive, ~3-5s) Topics: general (default), --news, --finance

4. xpoz_search.py — Social Media Search

Twitter/X, Reddit, and Instagram search via Xpoz SDK. 12 subcommands.

python3 ~/.claude/skills/omnisearch/scripts/xpoz_search.py SUBCOMMAND "query" [options]
Quick Example Purpose
... xpoz_search.py twitter "Iran conflict" -n 10 Twitter search
... xpoz_search.py twitter "query" --sort latest --min-likes 10 Filtered tweets
... xpoz_search.py twitter-user elikishtaini -n 10 User timeline
... xpoz_search.py twitter-thread 1234567890 Full thread
... xpoz_search.py twitter-hashtag IranStrikes -n 10 Hashtag search
... xpoz_search.py reddit "geopolitics Iran" -n 10 Reddit search
... xpoz_search.py reddit-sub worldnews -n 10 --sort hot Subreddit posts
... xpoz_search.py reddit-post abc123 Reddit post + comments
... xpoz_search.py reddit-trending Trending subreddits
... xpoz_search.py instagram "conflict" -n 10 Instagram search

Twitter filters: --sort (latest/popular/media), --min-likes, --min-retweets, --since/--until, --lang Reddit filters: --subreddit, --sort (hot/new/top/rising), --time (hour/day/week/month/year/all)

Provider Selection Guide

Task Best Approach
General web search omnisearch.py "query" (auto-routes to Brave)
Breaking news omnisearch.py "query" --news or brave_search.py "query" --news --freshness pd
Quick factual answer omnisearch.py "query" --answer or tavily_search.py "query" --answer
Twitter OSINT omnisearch.py "query" --social or xpoz_search.py twitter "query"
Reddit sentiment xpoz_search.py reddit "query" or omnisearch.py "query" --social --platform reddit
Research papers omnisearch.py "query" --academic (routes to Exa)
Company intelligence omnisearch.py "query" (auto-detects company signals)
Multi-source corroboration omnisearch.py "query" --parallel --verbose
Content extraction tavily_search.py extract URL or Firecrawl
Domain-restricted search tavily_search.py "query" --domains site.com or brave_search.py "query" --goggles ID

Omnisearch vs Exa vs Firecrawl

Need Best Tool Why
Google SERP surface Omnisearch (Brave) Exa has independent index
Social media search Omnisearch (Xpoz) Only tool with Twitter/Reddit
AI answer with citations Omnisearch (Tavily) Agent-native answers
Multi-provider parallel Omnisearch --parallel Dedup across providers
Semantic/neural search Exa directly Superior relevance ranking
Research papers Exa --category publication Academic index
Find similar pages Exa exa_search.py with a query describing the page /findSimilar is deprecated
Single page scrape Firecrawl scrape Cleanest markdown
Full site crawl Firecrawl crawl Link following
News freshness filter Omnisearch (Brave --freshness) Hour/day/week/month
Content extraction Tavily extract or Firecrawl Both work

Common Workflows

OSINT Research (WWW-style)

# Step 1: Multi-provider news sweep
python3 ~/.claude/skills/omnisearch/scripts/omnisearch.py "Iran infrastructure strike" --news --parallel --no-text

# Step 2: Social signal check
python3 ~/.claude/skills/omnisearch/scripts/xpoz_search.py twitter "Iran infrastructure" --sort latest -n 15

# Step 3: Extract best sources
python3 ~/.claude/skills/exa-search/scripts/exa_contents.py URL1 URL2 --highlights --max-chars 5000

Quick Answer Pipeline

# AI answer with sources
python3 ~/.claude/skills/omnisearch/scripts/omnisearch.py "who commands CENTCOM" --answer --json | jq '.answer'

Cross-Platform Corroboration

# Get the same story from 3+ sources
python3 ~/.claude/skills/omnisearch/scripts/omnisearch.py "Strait of Hormuz incident" --parallel --verbose

Social Media Monitoring

# Track a developing story on Twitter
python3 ~/.claude/skills/omnisearch/scripts/xpoz_search.py twitter "#IranStrikes" --sort latest -n 20

# Check analyst activity
python3 ~/.claude/skills/omnisearch/scripts/xpoz_search.py twitter-user elikishtaini -n 10 --json

API Key Setup

Three new keys in ~/.config/env/secrets.env:

export BRAVE_API_KEY="..."      # https://api-dashboard.search.brave.com
export TAVILY_API_KEY="tvly-..."  # https://tavily.com (free tier, no CC)
export XPOZ_API_KEY="..."       # https://www.xpoz.ai

Existing keys (already configured for cross-skill routing):

export EXA_API_KEY="..."        # https://dashboard.exa.ai
export FIRECRAWL_API_KEY="..."  # https://firecrawl.dev

All scripts gracefully error with a clear message when their key is missing. You can use any subset.

Reference Documentation

File Contents
references/brave-api-reference.md Brave Search API: endpoints, parameters, response schemas, LLM Context API
references/tavily-api-reference.md Tavily API: search, extract, answer, topics, domains
references/xpoz-api-reference.md Xpoz API: SDK methods, Twitter/Reddit/Instagram endpoints

Test Suite

python3 ~/.claude/skills/omnisearch/scripts/test_omnisearch.py --quick     # Offline + one per key
python3 ~/.claude/skills/omnisearch/scripts/test_omnisearch.py             # Full suite
python3 ~/.claude/skills/omnisearch/scripts/test_omnisearch.py --group router   # Classification tests
python3 ~/.claude/skills/omnisearch/scripts/test_omnisearch.py --group dedup    # Dedup tests
python3 ~/.claude/skills/omnisearch/scripts/test_omnisearch.py --verbose   # Detailed output
Group Tests Live API?
router classify_query (7 categories), select_providers (5 scenarios), routing_table_coverage No
dedup exact URL, normalized URL, title Jaccard, short titles, score sorting, mixed providers, URL normalize, Jaccard math No
format human-readable, markdown, JSON serialization, no-text mode No
brave web search, news, freshness, formatters Yes
tavily basic, advanced, answer, extract Yes
xpoz twitter search, reddit, user tweets Yes
integration auto-route, parallel, cross-skill exa, JSON CLI output Yes