Most AI crawlers do not execute JavaScript. GPTBot, ClaudeBot, PerplexityBot, and nearly every other AI agent fetch your raw HTML and move on — no rendering, no waiting for scripts. If your content only exists after JavaScript runs, AI systems see an empty shell where your site should be. Here's how to test whether that's happening to you, and the fixes ranked by effort.
Why AI crawlers skip JavaScript
Google spent a decade and enormous infrastructure building a rendering pipeline: Googlebot fetches HTML, queues the page for a headless Chrome render, executes scripts, and indexes the result. AI companies crawling for training data or answer citations have no such pipeline — rendering JavaScript at web scale is expensive, slow, and mostly unnecessary for their goal of harvesting text. So they take the raw server response and parse whatever HTML is in it.
The practical consequence: the AI web is the no-JavaScript web. A page that renders beautifully in a browser can be an empty <div id="root"></div> to the crawlers deciding whether you're citable.
Which sites are affected
| Site type | AI crawlers see |
|---|---|
| WordPress, static sites, most CMSs | ✅ Full content — HTML is server-generated |
| Next.js / Nuxt / SvelteKit with SSR or SSG | ✅ Full content — rendered at build/request time |
| Plain React / Vue / Angular SPA (client-side rendered) | ❌ Empty shell — content arrives via JS |
| Content behind "load more" / infinite scroll | ❌ Only what's in the initial HTML |
| JS-injected sections on otherwise server-rendered pages (reviews widgets, pricing tables from APIs) | ⚠️ Page visible, those sections missing |
The last row is the sneaky one: your page isn't invisible, but the part that answers the question — the pricing table your SaaS renders from an API, the specs pulled in client-side — is.
Test what AI actually sees (two minutes)
Method 1 — the terminal test:
curl -s https://yoursite.com/your-page | grep -i "a phrase from your main content"
If the phrase isn't in the response, it isn't in your server-rendered HTML, and AI crawlers can't read it.
Method 2 — the simulator: paste your URL into our AI Snippet Simulator. It fetches your server-rendered HTML exactly the way an AI crawler does, then reports what a model can read, how it would describe your page, and what it would cite. If it comes back with "too little readable text," you've found your problem — that verdict is the crawler experience of your site.
Method 3 — browser, JS off: disable JavaScript in DevTools and reload your page. What you see is roughly what AI sees.
Run the AI-Readiness Audit afterward — it checks rendered content along with robots.txt access, llms.txt, and schema in one pass.
Fixes, ranked by effort
1. You're on WordPress or a server-rendered stack: nothing to fix. Verify with the tests above, especially any JS-injected widgets, and move on.
2. Your framework supports SSR/SSG — turn it on. React → Next.js, Vue → Nuxt, Svelte → SvelteKit. Pre-rendering (SSG) is ideal for content pages: the HTML is complete at build time, fast for humans and fully legible to every crawler. This is the durable fix.
3. Can't adopt SSR? Pre-render the pages that matter. Services and middleware that serve pre-rendered snapshots to bot user agents (based on user-agent detection) exist precisely for this. It's a patch — snapshots go stale, and you're maintaining a parallel version of your site — but it beats invisibility. If you go this route, make sure the AI user agents (GPTBot, ClaudeBot, PerplexityBot, and the rest of the bot directory) are on the list of agents that receive snapshots; many prerender configs only include Googlebot and Bingbot.
4. At minimum, server-render the critical content. Even in an SPA, put the H1, the opening answer paragraph, and key facts in the initial HTML payload, and let JavaScript enhance from there. AI crawlers get the substance; users get the app.
Don't forget the metadata
JavaScript-injected <title> tags, meta descriptions, and JSON-LD schema have the same problem — if your structured data is added client-side (a common pattern with tag managers), AI crawlers and most validators never see it. Schema belongs in the server-rendered <head>. Check yours with the Schema Validator using the URL option, which reads the raw HTML like a crawler would.
The one exception: Google
Google AI Overviews inherit Googlebot's rendering pipeline, so JavaScript content that Google indexes can surface there. That's why some SPA sites appear in AI Overviews while being absent from ChatGPT and Perplexity — different pipelines, different capabilities. Treat Google as the exception and raw HTML as the baseline: server-render your content and every AI surface can read you; rely on rendering and you're visible to exactly one.