Cloudflare and OpenAI say a research pilot helped ChatGPT search discover and index “hundreds of millions” of new web pages. It does not mean your pages will be cited, only that more of them can be found. To benefit, let OpenAI’s search crawler in, keep your pages fresh and free of dead URLs, and track ChatGPT visits properly.
What changed
On October 15, 2026, Cloudflare and OpenAI published early results from a research pilot first announced in July. According to Search Engine Journal and IT Brief UK, the pilot used signals from about 1% of Cloudflare’s network, such as how recently a page changed and the quality of traffic it receives. Those signals helped OpenAI’s search crawler find pages it had not seen before, including useful pages that few other sites link to.
The signals also help the crawler skip waste. They flag dead URLs, redirects that dump everything on the homepage, and near-duplicate pages, and they tell the crawler when a page has changed enough to be worth a revisit. The companies are also testing signals that flag a single harmful page, such as a fake login form on an otherwise legitimate site.
What it does not mean
Discovery is not citation. OpenAI still evaluates and ranks the pages it indexes on its own, and the announcement gave no numbers for citations or referral visits. The post also says a signal about a page does not give a crawler permission to fetch it, and that the data is used for search and discovery, not for training models. Your access choices still apply.
| OpenAI bot | What it does | If you block it |
|---|---|---|
| OAI-SearchBot | Finds pages for ChatGPT search results | Your site can drop out of ChatGPT search results |
| GPTBot | Collects content that may be used for model training | Your content is signaled as not for training, search visibility is unaffected |
Source: OpenAI crawler documentation, as summarized by Search Engine Journal, October 20, 2026.
Why business owners should care
If customers ask ChatGPT for a plumber, a software tool or a product comparison, your business can only appear if the crawler knows your page exists. Practical pages are the ones this pilot targets: a menu and hours, a fee schedule, a user guide, a service-area page. Many small business sites are full of exactly these pages. It is the same crawl-access theme we covered in Meta’s new crawling terms.
What to do about it
- Check your robots.txt today. Make sure OAI-SearchBot is allowed on the pages you want found. Decide separately whether to allow GPTBot, since that one is about training.
- Review your CDN and firewall settings. If you use Cloudflare, open AI Crawl Control, which Cloudflare says is on every plan, and confirm you are not blocking a crawler you want.
- Clean up dead weight. Fix broken URLs, redirect loops and duplicate pages so the crawler spends its visit on pages that matter.
- Keep key pages current. Update prices, hours, fees and guides, and show the date. Freshness is one of the signals named in the pilot.
- Track ChatGPT visits properly. Search Engine Journal reported on October 15 that some ChatGPT links lacked UTM tags or referrers, so check your analytics before judging results.
If you want this handled as part of one plan, our SEO, AEO and GEO services cover crawl access, page quality and AI visibility together. If budget is your first question, read how much SEO costs in 2026.
The bottom line
More of the web is now visible to ChatGPT search, which makes crawl access and page quality more important, not less. Get the basics right first, and you can see how it all fits together at GCL Marketing. You can also browse more articles by topic.
Frequently Asked Questions
What did Cloudflare and OpenAI announce?
They shared early results from a research pilot covering about 1% of Cloudflare’s network. They say Cloudflare’s signals helped OpenAI’s search crawler discover and index hundreds of millions of new pages for ChatGPT search. The results were published on October 15, 2026, after the pilot was announced in July.
Does this mean my pages will appear in ChatGPT answers?
No. The announcement is about discovery, meaning ChatGPT search can find more pages. OpenAI still evaluates and ranks what it indexes, and no citation or referral numbers were reported. You still need accurate, useful, current pages that answer real customer questions to be chosen as a source.
Should I block or allow OpenAI’s crawlers?
It depends on your goal. OpenAI documents OAI-SearchBot as the crawler for ChatGPT search and GPTBot as the training crawler. Blocking OAI-SearchBot can remove your site from search results, while allowing it and blocking GPTBot keeps you visible without signaling permission for training.
Does Cloudflare share my page content with OpenAI?
Cloudflare says it uses web metadata, such as freshness and traffic quality signals, for search and discovery, not page content for model training. A signal about a page also does not grant a crawler permission to fetch it. Your existing access settings, including robots.txt, still decide what is allowed.
How do I know if ChatGPT is sending me traffic?
Check referral sources and UTM tags in your analytics. Search Engine Journal reported on October 15 that some ChatGPT links lacked both, so visits could be mislabeled, and one analyst said a fix began rolling out that day. Compare several weeks of data before drawing conclusions.