Playbook · Robots.txt blocking products / collections / entire store
Shopify robots.txt accidentally blocking products, collections, or entire store from Google.
Written for a Shopify DTC owner who still sells fine for buyers, but Google Search Console shows Blocked by robots.txt on product, collection, or homepage URLs — because live robots.txt Disallow rules hit /products, /collections, / (entire store), or a path typo — or you chased sitemap lag, password page, rich snippets, or hreflang instead. DIY-first: prove live robots.txt → fix Disallow / SEO-app overrides → retest Googlebot fetch. Distinct from sitemap.xml not updating, password page customization / password page stuck online, rich snippets missing, and hreflang wrong. Soft-link shopify-robots-txt-blocking-indexing when the bug is meta robots noindex / password leftover / Markets gates rather than catalog path Disallow. No earnings claims, no invented case studies.
Nico at Latch AI Ops · Sep 10, 2026 · ~12 minute read
Buy the 72-hour install — $997Robots / noindex indexingSitemap not updatingMerchant FAQ
01
Robots.txt blocking products/collections/store — not sitemap not updating, password page, rich snippets, hreflang.
Several “Google never sees our catalog” products can look alike. They do not share a fix. This page is only when robots.txt Disallow accidentally blocks products, collections, or the entire store from Google. Sitemap.xml not updating with new products/collections so Google misses pages is shopify-sitemap-not-updating. Password / coming soon page stuck or customization missing is password-page-stuck-online / shopify-password-page-customization-missing. Product rich snippets / JSON-LD missing is shopify-rich-snippets-missing. Markets hreflang alternate links wrong is shopify-hreflang-wrong.
Soft links when the bug is adjacent: broader robots.txt / noindex / password leftover / Markets gates blocking indexing (not specifically catalog path Disallow) is shopify-robots-txt-blocking-indexing. Broader sitemap missing products is shopify-sitemap-xml-missing-products. This playbook stays on live robots.txt Disallow on /products, /collections, or / ↔ GSC Blocked by robots.txt ↔ buyers still get 200.
02
Prove live robots.txt Disallow lines vs GSC crawl blocks and buyer 200s.
Open https://YOURSTORE.com/robots.txt. Read every Disallow: line for Googlebot / * user agents. Flag rules that match /products, /collections, /, or accidental wildcards. Confirm a sample PDP and collection URL still return 200 for a normal browser. In GSC → Pages, note Blocked by robots.txt counts and sample URLs.
Soft-check: if Online Store password is still enabled, leave this page and use password-page-stuck-online. Soft-check: if new URLs are missing from sitemap XML after publish (not Disallow), use shopify-sitemap-not-updating. Soft-check: if Rich Results Test fails Product JSON-LD, use shopify-rich-snippets-missing. Soft-check: if wrong locale URLs in SERPs, use shopify-hreflang-wrong.
03
Repair robots.txt.liquid / SEO-app Disallow; verify Googlebot can fetch catalog.
Edit theme robots.txt.liquid (or the SEO app that publishes robots). Remove Disallow lines that block /products, /collections, or the entire store. Keep Shopify-default safe Disallows for cart, checkout, account, search, and admin. Turn off SEO-app “block all” or staging Disallow presets left on after go-live. Wait for CDN cache, then re-fetch robots.txt and confirm the bad lines are gone.
In GSC URL Inspection, Test live URL on a priority product and collection. Request indexing only after robots allows crawl. Soft-link shopify-robots-txt-blocking-indexing when the root cause is meta robots noindex / password leftover / Markets domain gates rather than path Disallow on catalog.
04
You tested sitemap lag, password page, rich snippets, or hreflang instead.
Sitemap.xml not updating is shopify-sitemap-not-updating. Password page stuck / customization missing is password-page-stuck-online / shopify-password-page-customization-missing. Rich snippets missing is shopify-rich-snippets-missing. Hreflang wrong is shopify-hreflang-wrong. Broader robots / noindex indexing DIY is shopify-robots-txt-blocking-indexing.
The only honest test for this page: buyers get 200 on products/collections → live robots.txt Disallow still blocks those paths (or /) → GSC reports Blocked by robots.txt → after fix, robots allows crawl and Inspection can fetch. Anything else is a different product.
05
When to stay DIY vs pay $997.
Stay DIY if you can spend one afternoon on the list above: prove live robots.txt, remove bad Disallow / SEO-app overrides, retest Googlebot fetch in GSC. That is the whole playbook. No agency required. Short answers also live on the merchant FAQ. Related pages: robots / noindex indexing, sitemap not updating, rich snippets missing, hreflang wrong, password page stuck.
Pay for help when the work is not the Disallow line itself — it is connecting the stack you already have so crawl / robots / indexing exception alerts land in a channel you already check, and the live store is one workflow instead of a spreadsheet plus a Slack bot you forgot. That is the Latch AI Ops install.
What $997 USD one-time buys: lead capture on the live store, ops alerts in a channel you already check, and a checkout-ready workflow. 72 hours after collaborator access, not after payment. One live Shopify store. Access plus a delivery thread. Store URL collected at checkout. Not a SaaS seat, not an SEO retainer, not a ranking or indexation guarantee, not a brand or theme job.
Other shops' public ranges, not testimonials: a checkout / ops install typically runs $500–$2,500. A full ops consulting block typically runs $2,000–$5,000. $997 is the cheap end of that work, sold as a 72-hour install of three systems — not as an earnings number or ranking-lift claim.
What it does not buy: more organic traffic, recovered rankings, guaranteed Google indexation forever, or a case study. Latch AI Ops has not published customer proof because there are no customers yet. You are buying the install.
Checkout is on Whop. Operator is Nico. United States offer. You can revoke collaborator access after handoff. We do not take owner passwords.