Why 200 Million Websites Still Can’t Answer a Natural Language Question

▶ Watch (00:40)

Of the 200 million active websites, exactly four support a real conversational interface. Guha’s framing is precise: the data already exists. A vacation rental site has beach proximity. A real estate database has school districts and pool flags. The query interface is the gap, not the inventory.

“There is no way to do that query on that site.” — Ramanathan Guha

Schema.org, Guha’s own standard, is now on 70 million of those 200 million sites, and LLMs understand it because it saturates their training data. NLWeb builds on that coincidence to add a conversational layer without hiring AI engineers.

Live Demo: Empty Database to Conversational Interface in Under Two Minutes

▶ Watch (07:39)

Jennifer Marsman started the live RSS load demo (12:27) with an empty Qdrant database. She copied the RSS feed URL for Kevin Scott’s “Behind the Tech” podcast, ran python tools/db_load with the URL and a site label, and watched it fetch episodes, create embeddings, and write them to the local vector store. The server restarted. The same query she had run against the empty database returned real results.

Common Sense Media had a working proof of concept in under a week using the same flow. The entry point is one command, one RSS feed, and a config file.

How the Pipeline Avoids Hallucination

▶ Watch (17:52)

NLWeb does not use RAG. Guha’s reasoning: RAG makes things up 1 time in 1,000, and you cannot tell a retailer that 1 in 1,000 products will be fabricated. Instead, the pipeline runs two branches in parallel. One call goes to the LLM for query understanding, one to the vector database for retrieval. Results re-rank, and the system returns schema.org objects, each backed by a database identifier.

“So, you know there is no hallucination.” — Ramanathan Guha

Every returned item is provably in your catalog. The LLM ranks and explains; it does not invent. Google set the expectation that search answers in under 500 milliseconds. NLWeb targets the same bar.

Getting Started: Config, Data Loading, and Front Ends

▶ Watch (23:51)

Three config files control everything: embedding provider, language model, and vector database. Switching from Azure OpenAI to plain OpenAI is one line. Supported databases at launch are Azure AI Search, Qdrant, and Snowflake. Data loads through python tools/db_load, which fetches an RSS feed or JSONL file, converts items to schema.org format, and writes embeddings to the chosen store.

A “Hello World” path in the docs folder walks the full setup: clone, activate a virtual environment, fill in the .env template, and start the server. Deployment docs cover Azure via zip deploy; other clouds will follow.

Built-in MCP Server: Every NLWeb Endpoint Becomes Agent-Compatible

▶ Watch (43:45)

Every NLWeb deployment ships an MCP endpoint automatically. In the Claude Desktop MCP demo (46:00), Chelsea queried an NLWeb server from Claude Desktop, asking about spicy crunchy snacks. The server selected relevant sites (Delish, a Latin American recipe collection), fetched results as JSON, and returned a ranked answer. No extra plumbing required.

Guha’s framing for why this matters: NLWeb output has the same shape as a vector database query result, so NLWeb instances can chain together. One endpoint aggregates across many others, DNS-style. That structure is what Satya Nadella meant on stage when he talked about preventing aggregator lock-in.

Q&A

How does a single natural language query get decomposed into 40 or 50 model calls? Guha pointed to a ‘Life of a Chat Query’ document in the repo’s docs folder, which walks through the full decomposition step by step. ▶ 52:42

You said RAG hallucinates but NLWeb doesn’t — can you clarify how the architecture prevents fabricated results? Guha explained the ‘mixed mode programming’ pattern: the system keeps deterministic control over query flow, using the LLM only as an oracle for fine-grained subtasks rather than dumping all data into the model and hoping for the best. ▶ 53:51

Notable Quotes

There is no way to do that query on that site. Ramanathan Guha · ▶ 1:05

So, you know there is no hallucination. Ramanathan Guha · ▶ 16:53

because what we’re competing with here is search. Ramanathan Guha · ▶ 27:37

Key Takeaways

  • NLWeb adds a conversational interface to any site in minutes by treating schema.org structured data as the input format.
  • The search-inspired pipeline — not RAG — guarantees every result links to a real item in your database.
  • Each NLWeb deployment ships a built-in MCP server, making the site’s data available to any AI agent.