DuckDuckBot
DuckDuckGo's search / answer indexing crawler.
Quick answer
DuckDuckBot is DuckDuckGo's web crawler, built to improve DuckDuckGo's search results and offer a secure search experience to its users. It crawls the open web the way other major search engines do, rather than existing for AI model training or user-triggered fetches.
What it does
DuckDuckBot crawls publicly available web pages so DuckDuckGo can build and refresh the index behind its search results. DuckDuckGo describes the bot's job as constantly improving search results and offering a secure search experience, which is the same general function Googlebot or Bingbot serve for their respective search engines. DuckDuckBot follows the WWW::RobotRules standard for interpreting robots.txt, so a site that disallows it should stop being crawled going forward, with previously indexed pages gradually dropping out of DuckDuckGo's results over time. DuckDuckGo publishes DuckDuckBot's IP ranges in a JSON file, which lets a site owner check whether traffic claiming to be DuckDuckBot is genuine before deciding how to configure access for it.
Facts
Operator
DuckDuckGo
Robots token
DuckDuckBot
User agent string(s)
- DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html)
Purpose
Search / answer indexing
Respects robots.txt
Yes
IP ranges
https://duckduckgo.com/duckduckbot.json
From DuckDuckGo's documentation, checked Oct 2026.
Should you block it?
DuckDuckBot's purpose is close to Googlebot's or Bingbot's: it builds the index DuckDuckGo search results come from, so disallowing it removes a site from that search engine's organic results over time rather than affecting AI training or AI answer visibility specifically. A brand that wants to be discoverable through DuckDuckGo, which markets itself on privacy and has a meaningful user base of its own, should leave DuckDuckBot allowed the same way it would allow other major search crawlers. There's little reason to block DuckDuckBot unless a site wants to opt out of search indexing broadly, since DuckDuckGo doesn't document a separate AI-training use for this particular bot the way OpenAI or Anthropic do for theirs. The tradeoff here is standard search visibility, not the newer AI-answer considerations that apply to bots built specifically for generative AI products.
How to block it
Add this to robots.txt to block DuckDuckBot from crawling any page: User-agent: DuckDuckBot Disallow: / DuckDuckBot follows the WWW::RobotRules standard, so a disallow rule should stop future crawling of the blocked paths.
Frequently asked questions
What does DuckDuckBot do?
DuckDuckGo describes DuckDuckBot as its web crawler, used to constantly improve search results and offer a secure search experience, similar to how other search engines crawl the web.
Does DuckDuckBot respect robots.txt?
Yes. DuckDuckBot follows the WWW::RobotRules standard for interpreting robots.txt directives, the same general approach other major search crawlers use.
How can I verify traffic claiming to be DuckDuckBot?
DuckDuckGo publishes DuckDuckBot's IP ranges as a JSON file, so incoming requests can be checked against that published list.
Related
- AI crawler directory — every AI crawler's robots.txt token, purpose and facts in one place.
- Free AI crawler checker — check which AI crawlers a domain's robots.txt actually allows or blocks.
- Glossary — definitions of the AI-visibility terms that come up alongside crawler behavior.
Sources
- DuckDuckBot is DuckDuckGo's web crawler, used to improve search results, and it respects WWW::RobotRules. Source: https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot/ (checked Oct 2026)
MarketHQ tracks brand mentions across communities, news, blogs, social and AI answers, and turns them into gap analysis and action plans.