Meta’s updated Commercial Terms, effective October 30, 2026, let it use automated tools to crawl websites tied to businesses that use its products, even if they never buy ads. After backlash, Meta dropped the word “train” from the text, but the crawling permission stays and the terms do not mention robots.txt. Review your Meta crawler rules and server logs before the deadline.
What Meta changed
Search Engine Journal reports that Meta’s updated Commercial Terms and revised Self-Serve Ad Terms take effect on October 30, 2026. A new paragraph lets Meta, or entities acting for it, access, copy, index and periodically re-crawl domains and pages associated with a business’s products or services. The terms cover any business use of Meta’s products, from running ads to managing a Facebook Page, and continued use counts as agreement. The only way to reject them is to stop using Meta for business.
PPC Land notes that the terms do not define what “associated with your products or services” means, and that Meta can create test credentials to reach pages. The earlier March 2024 Commercial Terms had no crawling clause, and the August 2025 Self-Serve Ad Terms covered only advertisers’ domains.
Meta’s response, and why publishers are still uneasy
Meta spokesperson Andy Stone said the update is not about scraping news sites and is meant to pull accurate product information for ads and recommendations. On October 17, The Next Web reported that Meta rewrote the terms late on Friday and removed the word “train” from the list of uses. Meta also said it is not changing how its web crawlers work.
The News/Media Alliance told The Next Web the change only narrows how Meta can use the content, and that AI training is “still potentially on the table.” Its spokesperson also noted the terms do not say Meta will honor robots.txt. Meta says site owners can still use robots.txt to manage access, but that promise sits outside the contract language.
What robots.txt can and cannot do
Search Engine Journal lists the crawlers above and suggests a site could block Meta-ExternalAgent while allowing Meta-WebIndexer. PPC Land adds a catch: Meta-ExternalAgent uses one robots.txt token for both training and indexing, so you cannot allow one and refuse the other. Meta-ExternalFetcher and FacebookExternalHit may bypass robots.txt in some cases. Legal experts quoted by Search Engine Journal also warn that Meta could argue its contract overrides blocking signals, though no court has ruled on it.
What it means for SEO, AEO and GEO
Crawler control is now part of answer and generative visibility, not just a security chore. Meta AI search builds its own index through Meta-WebIndexer, so the pages Meta can read shape the answers it gives about your brand. That is why we treat SEO, AEO and GEO as one system. The same thinking applies to how Google is placing links in AI Mode, and to how Google handles PDFs in AI search: what machines can access decides what gets cited.
What to do before October 30
- Find out if you are covered. Check whether your business has a Facebook Page, Business Suite access or an ad account, and who on your team accepts Meta terms. Remember that continued use counts as agreement.
- Audit your robots.txt. Look for rules covering Meta-ExternalAgent, Meta-WebIndexer, Meta-ExternalFetcher and FacebookExternalHit, and note which ones you never set.
- Check your server logs. Filter by those four user agents to see which Meta bots already visit, how often, and which sections they request.
- Decide bot by bot. Blocking Meta-ExternalAgent may reduce training use but also indexing for Meta products, so tie the choice to your business goals, not panic.
- Review what is public. Experts warn test credentials could get past sign-up walls, so make sure private, paid or sensitive pages are not protected by a basic registration form alone.
- Write it down and re-check. Record your decision and the date, then review again after October 30, since Meta has already changed the wording once.
The bottom line
Meta wants clearer permission to crawl the sites behind its business users, and it has softened the wording without changing the core clause. Whatever you decide, make it a deliberate choice backed by logs and rules, not something you discover later.
If you want help auditing crawler access and AI visibility together, our SEO services cover search, answer and generative visibility as one framework, and you can see how it fits at GCL Marketing. You can also browse more articles by topic.
Frequently Asked Questions
What do Meta’s new crawling terms say?
Meta’s updated Commercial Terms, effective October 30, 2026, let Meta use automated tools to access, copy and re-crawl websites tied to a business’s products or services. After backlash, Meta removed the word “train” from the text. Businesses agree simply by continuing to use Meta’s business products.
Does blocking Meta in robots.txt stop the crawling?
Not necessarily. Meta says content owners can still manage access through robots.txt, but the new terms do not mention robots.txt. The News/Media Alliance notes the terms do not promise Meta will honor it, and crawlers such as Meta-ExternalFetcher and FacebookExternalHit may bypass robots.txt in certain cases.
Which Meta crawlers should site owners know about?
Four are worth knowing for site owners. Meta-ExternalAgent collects pages for training foundation models and improving products. Meta-WebIndexer builds the index behind Meta AI search answers. Meta-ExternalFetcher fetches pages at a user’s request. FacebookExternalHit creates link previews when someone shares your URL.
Should I block Meta-ExternalAgent?
It depends on your goals. Search Engine Journal suggests a site could block Meta-ExternalAgent while allowing Meta-WebIndexer, but PPC Land notes ExternalAgent uses one robots.txt token for both training and indexing. Blocking is a real trade-off. Check your server logs and business priorities before deciding.
How do Meta’s crawling terms affect SEO, AEO and GEO?
It makes crawler management part of AEO and GEO. Meta AI search is another answer engine that builds its own index, so your product pages, pricing and policies should be accurate and crawlable. Review robots.txt, log which Meta bots visit, and keep a dated record of your decisions.