Why Your Website Is Invisible to ChatGPT, Claude and Perplexity — And How to Fix It

When someone searches on Google, a bot crawls your page, reads the text, and adds it to an index. You optimize for that: keywords, backlinks, meta tags. You know this game.

AI search engines — ChatGPT, Claude, Perplexity — work differently. They send their own crawlers (GPTBot, ClaudeBot, PerplexityBot) to collect and train on web content. When a user asks "recommend a marketing agency in Munich," the model answers from what it has already read — not from a live search.

If your site was never readable to those crawlers, you simply do not exist in that answer. No matter how good your Google ranking is.

I audit websites for AI visibility. The pattern I see is consistent: businesses that rank well on Google often score close to zero on AI readiness. Not because the content is bad — but because the technical setup makes the content invisible.

Here is a concrete example. A business runs a React-based website. Google visits, renders the JavaScript, reads the full page — product descriptions, service pages, team info. The site ranks well.

GPTBot visits the same site. It receives this:

Zero content. Zero context. The crawler records an empty page and moves on. Your business is not in the training data, not in the AI's memory of your industry, and not in the answers it generates.

JavaScript-rendered sites are the most common case — but not the only one. Even sites with server-side content can have other problems: robots.txt that blocks AI bots, no sitemap, no llms.txt, and content buried in inaccessible DOM structures.

Search behavior is shifting. A growing share of commercial queries now go through AI interfaces: ChatGPT, Perplexity, Google AI Overviews, Claude. Users ask "find me a freelance designer in Berlin" or "best accounting software for German SMBs" — and get a direct answer with named providers.

If your site is invisible to AI crawlers, you are not in those answers. Your competitors — the ones whose sites happen to be server-rendered, whose robots.txt happens to allow AI bots, whose llms.txt happens to exist — are being recommended instead.

This is not a future problem. It is happening now. And it compounds: the longer AI models go without seeing your content, the more thoroughly your competitors' positioning gets embedded in the models' understanding of your industry.

Generative Engine Optimization (GEO) is the practice of fixing this. It is not the same as classic SEO — though many of the fixes improve both. GEO focuses specifically on making your content readable and citable by AI systems.

If your site uses React, Vue or Angular without SSR

This is the highest-priority fix. Migrate to server-side rendering. For React: Next.js with SSR or SSG (Static Site Generation). For Vue: Nuxt with SSR. For Angular: Angular Universal. If a full migration is not feasible immediately, add a prerendering layer (Prerender.io, Netlify Edge Functions) that serves static HTML to crawlers.

Check and fix robots.txt

Open yourdomain.com/robots.txt. Look for User-agent: * with Disallow: / — this blocks everything including AI bots. Look for explicit blocks: User-agent: GPTBot with Disallow: /. Remove any rules that block legitimate AI crawlers. If you want to allow all bots: User-agent: * followed by Allow: /.

Create an llms.txt file

Create a plain text file at /llms.txt. Include: your company name and one-sentence description, what services you offer, who you serve, where you operate (city, region, country), and links to your 5-10 most important pages. No special format required — plain readable text works. Some sites use Markdown formatting. The goal is to give AI systems a clear, concise briefing about your business.

Add or fix your XML sitemap

Most CMS platforms (WordPress, Webflow, Squarespace) generate sitemaps automatically. Check that yours exists at /sitemap.xml and is referenced in robots.txt. If you built a custom site, generate a sitemap with a tool like xml-sitemaps.com and host it at the root domain.

Expand readable content

Every key page (services, about, contact, location pages) should have substantial readable text in HTML — not just images and headers. Describe what you do, for whom, and why. This is good for classic SEO too — so there is no conflict with your existing optimization work.