Troubleshooting

Each section below is a problem customers actually run into, followed by its fix.

#The widget is not appearing

Work down this list in order. It is ordered by how often each one is the cause.

#1. Is the script on the page?

View source on your live site and search for crawlbrain.

Not there. The install did not take. Purge every cache in front of your site: your CMS cache plugin, your CDN, Cloudflare. Then check again. If you edited a theme file, confirm you edited the live theme and not a duplicate.

There. Move to step 2.

You can also run Search Widget → Install → Check installation, which does the same fetch from our side.

#2. Is there a JavaScript error?

Open the browser console and look for errors mentioning the widget.

A content security policy on your site can block a script from cdn.crawlbrain.com. If you run a CSP, add that host to script-src and connect-src.

#3. Is the placement selector matching anything?

This check applies only to the two search-bar placements.

Run this in the console:

document.querySelector('YOUR-SELECTOR')

null means the selector matches nothing. Fix it, or blank the field and let auto-detect try. See Placements.

If your search input is itself rendered by JavaScript after page load, the widget may look before it exists. Use the Floating button placement instead.

#4. Is it disabled on this page?

Check for data-mode="disabled" on the script tag for that template.

#5. Is your cache plugin mangling it?

If your optimiser combines or defers JavaScript, exclude the CrawlBrain script from it. Bundling a version-pinned CDN file into a local build breaks it.

The console says CrawlBrain: API key missing on script tag when the optimiser rewrote the script address and dropped the key. Put the key on the tag as data-key, which the WordPress plugin does from version 1.2.1, then purge the cache.

#6. Is this domain allowed?

The console says This widget key is not allowed on followed by your domain. The script works only on the website it belongs to and the domains listed under Search Widget → Install → Allowed domains. Add the domain there and reload the page.

#CrawlBrain cannot reach my site

Open Knowledge → Sync History. The status column tells you which case you are in.

403, or bot_challenge in the cause. Something is blocking the crawler. Copy the IP address at the top of that page and whitelist it in your firewall or security plugin, or in Cloudflare if your site runs behind it. This is the fix for the large majority of these tickets.

429. You are rate-limiting us. Set Crawl speed in Sync Settings to 1 page every 2 seconds, or every 5 seconds if that is not enough.

5xx. Your server erred, usually under load. Choose a slower Crawl speed and retry.

Timeout or connection refused. The server did not answer. If the site is reachable in a browser, a firewall is likely dropping our requests silently.

login_wall. The URL redirected to a login form. Content behind authentication cannot be indexed.

Use Inspect live on the row to see exactly what comes back right now, without changing your index. It is the honest answer to "but it works in my browser", because the crawler has no cookies and no session.

#The sync found no pages, or far too few

1. Check your sitemap. The health strip at the top of Sync Settings tells you what was found. An empty or missing sitemap means few pages.
2. Check your exclusions. An over-broad pattern like /* excludes everything.
3. Check for a redirect. If example.com redirects to www.example.com, those are different hosts and the crawl stops. Add the website using the URL your site actually serves.
4. Check noindex. If Skip noindex pages is on and your pages carry the tag, they are skipped correctly.
5. Check the page cap. See Plans for the limit on your account.

#Pages sync but the answers are empty or bad

This is a content problem far more often than a model problem. Work in this order.

#Read what got stored

Open Knowledge → Website, then the row menu, then View content.

Mostly navigation and footer links. Add selectors under Exclude HTML elements in Sync Settings. Start with nav, header, footer, then add your theme's sidebar and related-posts blocks.

Nearly empty. Your content renders client-side. Turn on Render JavaScript and re-sync, and expect the sync to take roughly three times as long.

Fine, but the answer is still wrong. Keep going.

#Look for competing pages

Two pages saying different things about the same topic produce a hedged answer. Old pricing pages and superseded docs are the usual culprits.

Find them and exclude them from AI: they stay searchable but stop being used as answer sources.

#Check whether your page actually says it

Open the page and read it as a stranger. Very often the page implies the answer without stating it, because everyone internally already knows.

A person reading the page can infer the missing step, but retrieval only finds what the text says outright. One explicit sentence usually fixes it.

#Write a snippet

A snippet is the most direct fix available. Title it with the exact question and answer it in three sentences, and it is indexed within a minute. See Snippets.

#Only then, change the model

Premium is worth trying for technical documentation, but no model will rescue thin content. See AI Model and Tone.

#Answers are in the wrong language

The answer language follows the visitor's question and browser, not your setting.

If a visitor asks in English and gets Spanish, check whether that page is stored with the wrong language on it. If your site has language subdirectories, consider one CrawlBrain website per language, so each index is clean.

Interface strings are separate and are set per language in Texts and Languages.

#The widget looks wrong on my site

Colors clash. Set them explicitly in Appearance rather than relying on Auto.

The dark theme picked wrong. Auto reads the page background. On a light content area inside a dark wrapper it can choose wrong. Set Light or Dark explicitly, and use data-theme per page where they differ.

The trigger sits badly next to your search bar. Use the spacing overrides rather than fighting your CSS. See Per-Page Overrides.

Broken on mobile only. Check it with the browser extension at phone width. The cause is usually a long button label wrapping.

#API returns 403

subscription does not include API access means Pro is not active on the account. It is checked live, so a cancellation takes effect on the very next request.

plan_limit means you hit a website or page allowance.

#API returns "invalid request"

The Api-Key header is missing, misspelled, or the key does not exist. Keys start with cb_.

Check you are not sending an agent key where an API key belongs. They are different credentials. See Authentication.

#The dashboard looks empty but I have data

Check the website selector in the top bar. Everything is scoped to one website, and the selector is the most common reason a screen looks blank.

#Syncing has stopped happening on its own

If a site has had no logins and no widget traffic for 15 days, nightly syncs pause so idle accounts do not burn AI credits. You get an email before it happens.

Any login or any widget activity resumes them. You can also press Sync now.

#Still stuck

Send support a screenshot along with the website name and roughly when it happened. If it is a crawl problem, the relevant row from Sync History tells us most of what we need.