GigaCommerce
GEO & AI search

Welcome-list robots.txt

Definition

A robots.txt posture that explicitly allows AI crawlers across discovery content — product pages, collections, guides, policies — while disallowing transactional and private paths. The deliberate opposite of the publisher default-deny.

Defined in Should You Block AI Crawlers? A Merchant Decision Guide.

Want to know where you actually stand?

Submit your store and get a /100 agentic-commerce readiness score across catalog, schema, product pages and checkout — plus three fixes you can ship yourself.