# DuckDuckBot

## Quick answer

DuckDuckBot is DuckDuckGo's web crawler, built to improve DuckDuckGo's search results and offer a secure search experience to its users. It crawls the open web the way other major search engines do, rather than existing for AI model training or user-triggered fetches.

## What it does

DuckDuckBot crawls publicly available web pages so DuckDuckGo can build and refresh the index behind its search results. DuckDuckGo describes the bot's job as constantly improving search results and offering a secure search experience, which is the same general function Googlebot or Bingbot serve for their respective search engines. DuckDuckBot follows the WWW::RobotRules standard for interpreting robots.txt, so a site that disallows it should stop being crawled going forward, with previously indexed pages gradually dropping out of DuckDuckGo's results over time. DuckDuckGo publishes DuckDuckBot's IP ranges in a JSON file, which lets a site owner check whether traffic claiming to be DuckDuckBot is genuine before deciding how to configure access for it.

## Facts

- Operator: DuckDuckGo
- Robots token: DuckDuckBot
- User agent string(s): DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)
- Purpose: search-index
- Respects robots.txt: yes
- IP ranges: https://duckduckgo.com/duckduckbot.json

## Should you block it?

DuckDuckBot's purpose is close to Googlebot's or Bingbot's: it builds the index DuckDuckGo search results come from, so disallowing it removes a site from that search engine's organic results over time rather than affecting AI training or AI answer visibility specifically. A brand that wants to be discoverable through DuckDuckGo, which markets itself on privacy and has a meaningful user base of its own, should leave DuckDuckBot allowed the same way it would allow other major search crawlers. There's little reason to block DuckDuckBot unless a site wants to opt out of search indexing broadly, since DuckDuckGo doesn't document a separate AI-training use for this particular bot the way OpenAI or Anthropic do for theirs. The tradeoff here is standard search visibility, not the newer AI-answer considerations that apply to bots built specifically for generative AI products.

## How to block it

Add this to robots.txt to block DuckDuckBot from crawling any page:

User-agent: DuckDuckBot
Disallow: /

DuckDuckBot follows the WWW::RobotRules standard, so a disallow rule should stop future crawling of the blocked paths.

## FAQ

### What does DuckDuckBot do?

DuckDuckGo describes DuckDuckBot as its web crawler, used to constantly improve search results and offer a secure search experience, similar to how other search engines crawl the web.

### Does DuckDuckBot respect robots.txt?

Yes. DuckDuckBot follows the WWW::RobotRules standard for interpreting robots.txt directives, the same general approach other major search crawlers use.

### How can I verify traffic claiming to be DuckDuckBot?

DuckDuckGo publishes DuckDuckBot's IP ranges as a JSON file, so incoming requests can be checked against that published list.

## Sources

- DuckDuckBot is DuckDuckGo's web crawler, used to improve search results, and it respects WWW::RobotRules. Source: https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot/