# OAI-SearchBot

OAI-SearchBot is OpenAI's crawler that builds the search index behind ChatGPT search. When someone asks a question, the answer is grounded in, and cites, pages this bot has indexed.

## Should you allow OAI-SearchBot?

**Allow it.** If OAI-SearchBot can't crawl you, ChatGPT search can't cite you, no matter how good your content is. Blocking it is the most common mechanical reason a brand is missing from AI answers.

## robots.txt

To allow OAI-SearchBot everywhere:

```text
User-agent: OAI-SearchBot
Allow: /
```

To block it:

```text
User-agent: OAI-SearchBot
Disallow: /
```

Bots follow the most specific `User-agent` group that matches them. If you have a `User-agent: OAI-SearchBot` group, the rules under `User-agent: *` no longer apply to it. Matching on the token is case-insensitive.

## Identifying OAI-SearchBot

OAI-SearchBot identifies itself with `OAI-SearchBot` in its `User-Agent` header. It **does not run JavaScript**: it reads the raw HTML only. Content your site adds with JavaScript is invisible to it.

OpenAI publishes the IP ranges OAI-SearchBot crawls from at [openai.com/searchbot.json](https://openai.com/searchbot.json). Anyone can fake a user agent, so a hit claiming to be OAI-SearchBot from outside those ranges is spoofed. LLMentions' `bots_report` checks this for you.

## Check your site

```bash
llmentions audit example.com            # is OAI-SearchBot allowed by robots.txt, and what does it see?
llmentions bots /var/log/nginx/access.log   # did OAI-SearchBot really visit, and is it genuine?
```

Or identify any user agent over REST (free): `GET https://api.llmentions.com/v1/bots/lookup?ua=OAI-SearchBot`.

## Other OpenAI bots

- [GPTBot](/bots/gptbot/): AI training crawler
- [ChatGPT-User](/bots/chatgpt-user/): live AI user fetch

---
https://llmentions.com/bots/oai-searchbot/
