CatalogScan

How-To Guide

How to Create (and Verify) a Product Feed in Shopify

The surprising answer: you do not need to create anything. Shopify already built your product feed. The real question is whether it is actually accessible — to Google, to ChatGPT, and to the other AI shopping agents that drive purchase decisions.

TL;DR Every Shopify store has a built-in product feed at /products.json. No app or plugin is required. The feed is blocked far more often than merchants realize — by Cloudflare WAF rules, headless storefront misconfigurations, or Shopify's own password protection. This guide walks through 5 steps to verify it is working and fix it if it is not. Time required: 5 minutes.

The 5-step verification checklist

1

Open your product feed in an incognito browser

Navigate to https://yourdomain.com/products.json in an incognito window (important — cached sessions can mask redirects). You should see raw JSON starting with {"products":[.

✓ Expected: JSON response with products array

✗ Problem: Password page, 403, 404, or an HTML interstitial

If you see the Shopify storefront password page, your store is in pre-launch mode. Go to Shopify Admin → Online Store → Preferences → Disable password. If you see an error page, proceed to Step 2.

2

Test the feed with a bot user-agent

AI shopping agents identify themselves with a user-agent string like GPTBot/1.0. Cloudflare and other WAFs treat these differently from browser requests. Run this from your terminal:

curl -sI -A "GPTBot/1.0" https://yourdomain.com/products.json

✓ Expected: HTTP/2 200 and content-type: application/json

✗ Problem: HTTP/2 403, HTTP/2 401, or content-type: text/html (Cloudflare challenge page)

If the browser shows 200 but this curl returns 403 or HTML, Cloudflare's Bot Fight Mode is the culprit. See Step 4 for the fix.

3

Verify the product count

A feed that loads but returns 0 products is just as broken as a blocked feed. Check the count:

curl -s "https://yourdomain.com/products.json?limit=250" | \
  python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d['products']), 'products on page 1')"

✓ Expected: number close to your actual published product count (up to 250)

✗ Problem: 0 products, or far fewer than your catalog size

Zero products usually means you are on a headless storefront (Hydrogen on Vercel, Next.js, custom React) that does not proxy /products.json to your Shopify back-end. The new server returns an empty response instead of routing the request to Shopify's servers.

4

Check robots.txt for AI crawler blocks

Visit https://yourdomain.com/robots.txt and search for lines mentioning GPTBot, ClaudeBot, PerplexityBot, or blanket Disallow: / rules applied to User-agent: *.

User-agent: GPTBot
Disallow: /        # ← This blocks all ChatGPT crawling — remove or limit to /checkout/

Shopify's default robots.txt (the one it generates) does not block AI crawlers. But custom Liquid robots.txt files added by some apps or theme developers do. If you have a custom robots.txt, review it carefully.

5

Fix identified blockers

Cloudflare Bot Fight Mode blocking bots: In Cloudflare Dashboard → Security → Bots, disable Bot Fight Mode or create a Skip WAF rule that exempts specific AI user-agents. See our Cloudflare AI crawlers guide for the exact rule syntax.

Headless storefront not proxying /products.json: In your Next.js or Remix project, add a proxy route:

// pages/api/products.js (Next.js pages router)
export default async function handler(req, res) {
  const shopifyUrl = `https://${process.env.SHOPIFY_STORE_DOMAIN}/products.json`;
  const upstream = await fetch(shopifyUrl + '?' + new URLSearchParams(req.query));
  const data = await upstream.json();
  res.status(200).json(data);
}

Then add a rewrite in next.config.js to map /products.json to this API route, or handle it in middleware.

Shopify password protection: Disable in Shopify Admin → Online Store → Preferences → Password page. There is no way to make just the feed public while keeping the storefront gated.

What about custom product feeds?

If you need a feed in a specific format — Google Shopping XML, Meta Catalog, or a custom CSV — you have two options:

  • Shopify channels (Google & YouTube, Facebook & Instagram): These apps generate the appropriate feed format and sync it directly to the relevant platform. They work in parallel with /products.json and do not replace it.
  • Feed apps (DataFeedWatch, Omnivore, Nabu): Third-party apps that pull from Shopify's API and output a feed in any format you need. Useful for complex attribute mapping or multi-channel feeds.

For AI shopping agent discoverability, custom feed formats are not helpful — agents access /products.json directly. Focus on keeping the built-in feed public and complete.

How CatalogScan automates this check

Running these 5 steps manually takes about 5 minutes and requires access to a terminal. CatalogScan does all of them in one scan — including the bot user-agent test, product count check, robots.txt parse, and Cloudflare detection — and returns a scored report with prioritized fixes. It also runs the scan from a real server (not localhost), catching network-level blocks that curl from your own IP might not trigger.

Paste your store URL — full audit in under 2 minutes, no account required.

Check my product feed →

Common questions

Do I need to install an app to create a product feed in Shopify?

No. Shopify creates /products.json automatically for every store on the Online Store channel. No app is needed to create the feed. Apps are only needed if you want a different format (Google Shopping XML, Meta Catalog) or if your feed needs advanced attribute mapping. For AI agent discoverability, the built-in feed is what matters.

How do I add products to my Shopify product feed?

Products appear in the feed automatically when they are published in your Online Store sales channel. If a product is missing from the feed, check that it is not set to "Draft" status and that it is included in the "Online Store" sales channel (Shopify Admin → Products → click the product → Sales channels). Unpublished or hidden products do not appear in /products.json.

Can I control which products appear in the feed?

You can only control which products appear by publishing or unpublishing them in the Online Store channel. There is no built-in way to exclude specific products from /products.json while keeping them published. If you need fine-grained control, some feed apps offer filtering by tag, collection, or metafield — but those apps output to their own feed URLs, not to the standard /products.json.

Related reading