LLMentions
AI crawlers · Bingbot

Bingbot: Microsoft's search engine crawler

Bingbot is Microsoft's search engine crawler. What it does, whether to allow it, robots.txt rules and how to verify real Bingbot traffic.

↓ Markdown for agents
Vendor
Microsoft
Type
Search engine crawler
Runs JavaScript
Yes
IP ranges published
Yes

Bingbot is Microsoft's search crawler. It indexes pages for Bing, and the Copilot answers built on it.

Should you allow Bingbot?

Allow it. Bingbot powers Bing, and the Copilot answers built on it. Blocking it removes you from that search engine.

robots.txt

To allow Bingbot everywhere:

text
User-agent: Bingbot
Allow: /

To block it:

text
User-agent: Bingbot
Disallow: /

Bots follow the most specific User-agent group that matches them. If you have a User-agent: Bingbot group, the rules under User-agent: * no longer apply to it. Matching on the token is case-insensitive.

Identifying Bingbot

Bingbot identifies itself with Bingbot in its User-Agent header. It renders JavaScript, but rendering is deferred and costly, so server-rendered content is still read faster and more reliably.

Microsoft publishes the IP ranges Bingbot crawls from at www.bing.com/toolbox/bingbot.json. Anyone can fake a user agent, so a hit claiming to be Bingbot from outside those ranges is spoofed. LLMentions' bots_report checks this for you.

Check your site

terminal
llmentions audit example.com            # is Bingbot allowed by robots.txt, and what does it see?
llmentions bots /var/log/nginx/access.log   # did Bingbot really visit, and is it genuine?

Or identify any user agent over REST (free): GET https://api.llmentions.com/v1/bots/lookup?ua=Bingbot.