A practitioner's checklist for making sure your framework's biggest strengths don't quietly cost you search and AI visibility.
Key takeaways
- Google indexes JavaScript in two waves. Content that only appears after client-side execution can sit in a render queue for hours or days before Google ever sees it.
- Most AI crawlers, including GPTBot, ClaudeBot, and PerplexityBot, do not execute JavaScript at all. They only ever see your initial HTML response.
- A page can rank first on Google and be a completely blank shell to ChatGPT, Claude, and Perplexity at the same time.
- Hydration mismatches can silently strip server-rendered content and replace it with a loading state that bots capture before the real content re-renders.
- The fix for nearly every issue in this article is the same principle: render core content, metadata, and structured data server-side, and reserve client-side JavaScript for enhancement, not for anything you need indexed.
On this page
The Modern Web Dilemma: Developer Agility vs. Search Indexation
Modern frontend frameworks exist to make developers faster. React, Vue, and their meta-frameworks let a small team ship complex, interactive interfaces without hand-rolling every page. That agility comes from shifting work that used to happen on the server onto the client, or onto a build step, or onto a rendering queue somewhere in between. Search crawlers were not built with that shift in mind.
The business cost shows up quietly. A product page that renders perfectly in a browser can sit unindexed for days because its content only exists after JavaScript runs. A pricing update pushed at 9am can still be showing yesterday's number in search results a week later. Crawl budget gets spent re-fetching JavaScript bundles and API calls instead of discovering new pages. None of this throws an error. It just slowly erodes visibility until someone notices organic traffic has stalled and starts trying to figure out why.
How Search Engines Process JavaScript: Wave 1 vs. Wave 2
Google's indexing pipeline was long described as two waves, and the mental model still holds even as the underlying system has evolved. In Wave 1, Googlebot crawls the raw HTML response, indexes whatever content is immediately available in that markup, and extracts links to queue for further crawling. Nothing here requires JavaScript execution.
Wave 2 is where JavaScript-dependent content gets its chance. Pages that need client-side execution are queued into Google's Web Rendering Service, which runs the JavaScript once compute resources are available and captures the resulting DOM. This queue is not instant. Render delays ranging from hours to days are common, and for anything time-sensitive, a price, a stock status, a limited-time offer, that delay can mean Google is indexing information that is already stale by the time it gets processed.
The AI Crawler Blind Spot: Why LLMs See Even Less Than Googlebot
This is the part most JavaScript SEO guides still miss. Everything above assumes Google eventually renders your JavaScript. Most AI crawlers never do.
GPTBot, ClaudeBot, PerplexityBot, and OAI-SearchBot fetch the raw HTML response and extract whatever text is already in that markup. They do not execute JavaScript, wait for a render queue, or retry after a client-side fetch resolves. Crawler-log analyses covering hundreds of millions of GPTBot requests have found no evidence of JavaScript execution at all, and separate research has found GPTBot and ClaudeBot occasionally download JavaScript files without ever running them.
The one meaningful exception is Google-Extended, the crawler that feeds Gemini and Google's AI Overviews. Because it inherits Googlebot's own rendering infrastructure, it can see JavaScript-rendered content the same way regular Google Search does. Microsoft's Copilot leans on Bing's index, and Bingbot's JavaScript rendering is more limited than Google's. Every other major AI crawler in wide use today reads only what arrives in the initial HTML response.
The practical result is a visibility split that did not exist a few years ago. A client-rendered single-page application can rank first on Google, where the Web Rendering Service eventually catches up, while remaining a completely empty shell to ChatGPT, Claude, and Perplexity. One 2026 industry analysis estimated that around 42% of JavaScript-rendered content never gets indexed by AI systems at all, a scale of blind spot that a purely Google-focused audit will never surface.
This changes how you should prioritize fixes. It is no longer enough to confirm a page eventually gets indexed by Google. If the goal includes being cited in AI answers, the only test that matters is whether the content is present in the raw server response, with zero JavaScript execution involved, because that is the exact condition every major AI crawler operates under.
Rendering Architecture: How Search Bots Experience Your Stack
Client-Side Rendering (CSR): High Risk for Critical Content
A CSR application typically serves a near-empty HTML shell, often literally <div id="root"></div>, and depends entirely on JavaScript execution to build out the actual page. For Googlebot, that means every page is a Wave 2 candidate by default. For AI crawlers, it means the page is functionally invisible, since there is nothing in that initial response to extract.
CSR also introduces failure modes that have nothing to do with whether a bot can technically run JavaScript. API timeouts during the render process, third-party scripts that block the main thread, and render queues that simply do not wait long enough for slow client execution can all leave a bot capturing an incomplete or empty page, even when the site works fine for a human visitor on a fast connection.
Server-Side Rendering (SSR) and Static Site Generation (SSG): The Production Standards
Pre-rendering, whether on the server per request (SSR) or at build time (SSG), removes the Web Rendering Service queue from the equation entirely for Google, and it is the only approach that reliably works for AI crawlers as well, since the content is already present in the initial HTML.
SSR generates the full HTML for each request as it comes in, which suits pages with frequently changing, personalized, or real-time content. SSG pre-builds static HTML at deploy time and serves it from the edge, which suits content that changes infrequently and benefits from being as fast as possible with no server compute per request.
Hybrid Rendering and Incremental Static Regeneration (ISR)
Modern meta-frameworks like Next.js App Router and Nuxt blend these models. Incremental Static Regeneration lets a page be built statically but revalidated on a schedule or on demand, without a full rebuild of the entire site. Handled correctly, this gives you SSG-level crawler visibility with content that stays reasonably fresh.
Handled incorrectly, it can serve a stale or partially-unrendered shell to bot user agents if cache-control headers and revalidation windows are not configured deliberately. Any hybrid setup should be tested specifically with a bot-facing request, not just a browser request, to confirm the cached response actually contains the content you expect.
Hydration Failures: The Underdiagnosed Visibility Killer
What Hydration Is and How It Breaks Search Equity
Hydration is the process of attaching client-side JavaScript event listeners and state to HTML that was already rendered on the server. It is what turns static-looking markup into an interactive application without a full re-render from scratch. When the DOM the client expects does not match the DOM the server actually sent, frameworks typically respond by discarding the mismatched portion and re-rendering it from the client, a full-tree re-render sometimes called DOM thrashing.
Content Drops and DOM Node Overwrites
Poorly handled hydration can strip out server-rendered elements, body copy, headings, specification tables, and replace them with a temporary loading state before the real content re-renders on the client. Search bots that capture the page during that intermediate window record the empty or loading state, not the actual content, even though a human visitor would never notice the flicker.
Client-Side Overwrites of Critical Head Metadata
The same problem gets worse when it hits your <head>. If the raw HTML ships a generic fallback <title>, <meta name="description">, and <link rel="canonical">, and the real, page-specific values are only set client-side inside a useEffect hook or a client-side router, Google frequently indexes the fallback text instead of the dynamic values you actually wanted. AI crawlers, which never run that client-side code at all, will see only the fallback, every time.
The Production JavaScript SEO Checklist
1. Crawlability, Resources, and Bot Directives
Check your robots.txt for accidental blocks on .js, .css, or backend API endpoints. Blocking an external API domain, for example api.example.com, is a common mistake that leaves the Web Rendering Service with a blank page to render, because the data it needs never loads. Also confirm HTTP status codes behave correctly: an unauthenticated client route should not return a 200 OK wrapped around what is functionally a client-side 404 or a client-side redirect, since that tells crawlers a broken page is healthy.
2. Link Extraction and DOM Discovery
Crawlers, both Googlebot and AI crawlers, discover new pages primarily by extracting the href attribute from native HTML <a> tags. This is the single rule most JavaScript navigation bugs violate. Watch specifically for:
- A
<span onClick={goToPage}>or a<button>used for page navigation instead of an anchor tag <a href="javascript:void(0)">, which gives a crawler nothing to follow- Hash-based routing (
/#/category) instead of the HTML5 History API (pushState), which produces URLs crawlers may not treat as distinct, indexable pages
3. Progressive Loading, Infinite Scroll, and Pagination
Bots do not scroll, hover, or click a "Load More" button. Any content gated behind those interactions is content a crawler will never see. Infinite-scroll interfaces should always ship standard fallback pagination links, for example <a href="?page=2">, alongside the scroll-based experience, so the same content is reachable through a plain, crawlable URL.
4. Structured Data and Schema Resilience
JSON-LD structured data should be baked directly into the initial server-side HTML response, not generated dynamically through a third-party tag manager or injected by a delayed client script. Schema that only appears after client-side execution is subject to the exact same Wave 2 delay as any other JavaScript-dependent content for Google, and it is invisible to AI crawlers entirely.
Diagnostic Workflow: How to Audit and Debug JavaScript Issues
View Source vs. Inspect Element (The Baseline Test)
These are two different documents, and confusing them is the most common mistake in JavaScript SEO audits. View Source (Ctrl+U) shows the raw server response exactly as a non-rendering crawler receives it. Inspect Element, or the DevTools Elements panel, shows the hydrated DOM after JavaScript has run. Any core content or metadata missing from View Source depends entirely on JavaScript execution to exist, which means it is invisible to every AI crawler in current use.
Disabling JavaScript in Browser DevTools
In Chrome DevTools, open Settings, find the Debugger section, and check "Disable JavaScript," then reload the page. What breaks tells you exactly what a non-executing crawler experiences. Can you still read the core content? Are links clickable? Can you reach primary navigation? If the answer to any of these is no, that content or navigation path does not exist for GPTBot, ClaudeBot, or PerplexityBot.
Terminal and Command-Line Inspection (curl)
To see exactly what payload a crawler receives, request the page with a bot user agent:
# Repeat with a GPTBot or ClaudeBot user agent string to check AI crawler visibility specifically,
# since they will not execute anything this request returns.
Review the response headers for X-Robots-Tag, confirm the HTTP status code is what you expect, and check the initial HTML size. A page that should have a full article or product description but returns only a few kilobytes of markup is very likely shipping an empty shell to any crawler that does not execute JavaScript.
Headless Crawler Comparisons (HTTP vs. Rendered)
Tools like Screaming Frog and Sitebulb support running two crawls of the same site, one using the default HTTP or text-based engine and one using full Chrome rendering. Compare the two crawl exports for word count discrepancies, missing internal links, and stripped structured data. The gap between them is, in practical terms, the gap between what an AI crawler sees and what Googlebot eventually sees.
Quick-Reference: Common Pitfalls and Technical Fixes
| Architectural Pitfall | Search Engine Impact | Engineering Fix |
|---|---|---|
| Clickable <span> or <button> navigation | Crawler stops; deep URLs remain orphaned | Replace with a standard <a href="/target"> |
| Client-only canonical injection | Wrong URL indexed, or canonical loops | Render <link rel="canonical"> server-side |
| Lazy-loaded text behind interaction | Text is omitted from search indexing | Render primary copy in the initial DOM; lazy-load images only |
| Blocked API routes in robots.txt | WRS fails to fetch content, leaving pages blank | Allow bot access to all rendering endpoints |
| Content only rendered via client-side JS | Invisible to GPTBot, ClaudeBot, and PerplexityBot entirely | Serve core content and metadata in the initial server response |
Engineering Search-Ready Web Applications
Balancing Performance, Usability, and Crawl Efficiency
Modern frontend frameworks and search visibility are not in conflict. The principle that resolves nearly every issue in this article is the same one: enforce server-first rendering for core content, primary navigation, and structured metadata, and reserve client-side JavaScript for the interactivity layered on top of that, not for anything you need indexed or cited.
Build Your Technical Search Foundation with Digital Appears
Eliminating rendering bottlenecks, solving complex crawl traps, and designing search-resilient frontend systems requires hands-on execution. If your team is running a headless stack, migrating to a modern JS framework, or troubleshooting persistent indexation drops, partner with Digital Appears for practitioner-led technical SEO and search architecture audits built for sustained growth.
Shishir Goel
Technical SEO practitioner at Digital Appears, focused on rendering architecture, crawl budget, and search-resilient frontend systems.
Frequently Asked Questions
Do ChatGPT, Claude, and Perplexity render JavaScript the way Google does?
No. GPTBot, ClaudeBot, and PerplexityBot fetch the raw HTML response and do not execute JavaScript at all. Google-Extended, which feeds Gemini and AI Overviews, is the exception, since it inherits Googlebot's rendering infrastructure.
Why does my page rank on Google but never get cited by ChatGPT or Perplexity?
The most common cause is client-side rendering. Google can eventually render your JavaScript through its Web Rendering Service, but AI crawlers do not execute JavaScript, so they only ever see whatever exists in the initial HTML response. If your core content is injected after the page loads, Google may index it while AI systems see an empty shell.
How long does Google's Wave 2 rendering actually take?
It varies, and Google has not published a fixed SLA. Delays ranging from hours to several days are common depending on crawl budget and site-wide rendering demand, which is one reason time-sensitive content should not depend on client-side rendering.
Is server-side rendering required, or can prerendering services fix a CSR site?
A prerendering or dynamic rendering layer that serves a fully-rendered HTML snapshot to bot user agents can work as a stopgap for an existing CSR application. For new builds, SSR, SSG, or a hybrid model is the more durable foundation, since it removes an extra moving part that has to be maintained and kept in sync with the live site.
Does using Next.js App Router or Nuxt automatically solve JavaScript SEO?
It gives you the tools to solve it, not a guarantee. Incremental Static Regeneration and hybrid rendering can deliver SSG-level crawler visibility, but only if cache-control headers, revalidation windows, and metadata rendering are configured deliberately. A misconfigured hybrid setup can still serve a stale or unrendered shell to bots.
Not sure what your own site looks like to a non-rendering crawler?
We run rendering and hydration audits as part of every technical SEO engagement, checked against both Googlebot and the AI crawlers that don't execute JavaScript at all.
Talk to Digital Appears about a technical SEO audit →

Add a Comment