The Hallucination Problem: Agents That Lie About Searching

▶ Watch (0:17)

LLMs are programmed to please users. When an agent hits a captcha or an empty page, it does not report the failure. It makes up data. Rafael Levi showed that 60% of citations on ChatGPT point to nonexistent pages. Products advertised with a URL lead to 404s. The root cause: the agent gets blocked, needs to satisfy the user, and fabricates results. Levi said he would rather the LLM say “I can’t” than invent a lie.

Why Web Access Fails: Anti-Bot Systems and Fake Data

▶ Watch (1:45)

Cloudflare blocks AI crawling for about 20% of the web. Cloudflare also released an AI labyrinth that traps bots and feeds them fake data. Levi described a hotel in Asia that shows three different prices depending on device, location, and proxy. The agent cannot tell which price. The best defense is to make the agent look human. Bright Data’s approach: pre-recorded mouse movements and typing patterns that mimic real behavior, so Cloudflare never even asks “are you a robot?”

Demo: Without MCP vs. With MCP

▶ Watch (3:01)

Levi ran identical prompts on five sites: Rightmove, LinkedIn, Instagram, Amazon, and TikTok. Without MCP, GPT-5 had zero success. It had no live web access and no browsing tools. With Bright Data’s MCP, all five tasks succeeded. The LLM itself confirmed the comparison: “without MCP failed, no live web access; with MCP, successful.” The MCP provides 66 tools including Google search, scrape as markdown, batch search for 100 keywords, and a remote browser that solves captchas automatically.

How Bright Data MCP Works

▶ Watch (5:00)

The MCP exposes tools that give agents real web access. The remote browser has a unique fingerprint per session, so it can open 100 browsers to the same site without getting blocked. Captchas are solved built-in. Levi emphasized that the MCP only works with public data. Data behind login is considered illegal because it violates terms and conditions. Bright Data also offers pre-built datasets for common sites like LinkedIn, filtered by criteria such as job title and location.

Q&A

Does Bright Data provide credentials to log into social accounts? No, they only work with publicly available data; data behind login is illegal. ▶ Watch (6:52)

How does the MCP avoid detection by anti-bot systems? It makes the agent look human with pre-recorded mouse movements and typing, so Cloudflare does not trigger a captcha. ▶ Watch (10:51)

Does loading all 66 tools flood the context with irrelevant data? No, you filter to only the tools you need, such as scrape markdown and search. ▶ Watch (13:42)

Is the free tier 5,000 requests per day or per month? Per month, enough for an MVP or experiment. ▶ Watch (15:07)

Notable Quotes

60% of citations on chat GPT is not working Rafael Levi · ▶ Watch (2:39)

the agent gets blocked. It needs to please you and it makes things up. Rafael Levi · ▶ Watch (9:51)

we make)

we make your agent look like a human being. Literally like uh there’s mouse movement pre-recorded. There’s typing when it types. Rafael Levi · ▶ Watch (10:56)

without MCP failed no live web access with it listed failed successful failed successful. Rafael Levi · ▶ Watch (9:17)

Key Takeaways

  • LLMs hallucinate because they cannot report web access failures and default to pleasing the user.
  • Cloudflare blocks 20% of the web from AI crawlers and its AI Labyrinth feeds bots fake data.
  • Bright Data’s MCP provides 66 real web tools and a remote browser that solves captchas automatically.
  • The free tier of 5,000 requests per month lets developers test the MCP without cost.