AlfredSearchBot
You're seeing our crawler in your server logs? Great. We're building a search engine that doesn't spy on people. Here's everything you need to know.
What is Alfred Search?
Alfred Search is a privacy-first, AI-native search engine built by GoSiteMe. Unlike Google, Bing, or even DuckDuckGo, we build our own independent web index using AlfredSearchBot — and we do it ethically.
No tracking. No cookies. No fingerprinting. No ad profiles. No filter bubbles. Just answers. We believe the web deserves a search engine that works for people, not on people.
About Our Crawler
AlfredSearchBot crawls publicly accessible web pages to build Alfred Search's independent web index. Here's our user agent string:
Respects robots.txt
We fully obey your robots.txt directives. Block us with User-agent: AlfredSearchBot and we'll never visit again.
Rate Limited
Maximum 1 request per second per domain. We also honor your Crawl-delay directive. We're a polite guest.
Depth Limited
We crawl to a maximum depth of 3 links from seed pages. We won't spider your entire site unless you invite us to.
HTML Only
We only request text/html pages. No images, PDFs, videos, or binaries are downloaded. Minimal bandwidth impact.
Control How We Crawl Your Site
Add these directives to your robots.txt file to control AlfredSearchBot:
# Block AlfredSearchBot entirely: User-agent: AlfredSearchBot Disallow: / # Allow crawling but set a crawl delay: User-agent: AlfredSearchBot Crawl-delay: 5 Allow: / # Allow crawling except private areas: User-agent: AlfredSearchBot Disallow: /admin/ Disallow: /private/ Allow: /
How Alfred Search Compares
| Feature | Bing | DuckDuckGo | Alfred Search | |
|---|---|---|---|---|
| Zero tracking | * | |||
| Own web index | (uses Bing) | |||
| AI instant answers | ||||
| No ad profiles | ||||
| Self-hostable | ||||
| Transparent ranking | ||||
| Deep research mode | ||||
| Open-source friendly | Partial |
* DuckDuckGo has improved privacy but still relies on Microsoft's Bing index and shows tracking-based ads.
For Webmasters & Developers
Submit Your Site
Want your pages in Alfred Search faster? Contact us to add your domain to our priority crawl queue. It's free.
Search API Access
Integrate Alfred Search into your own apps. Our REST API is simple, powerful, and comes with SDKs for Node.js, Python, and PHP.
White-Label Search
Run Alfred Search under your own brand. Perfect for enterprises, schools, and organizations that need private, internal search.
Try Alfred Search Right Now
Search the web without being watched. It's that simple.
Technical Specifications
Bot Name: AlfredSearchBot
Version: 1.0
Operator: GoSiteMe (gositeme.com)
Purpose: Web indexing for Alfred Search
Crawl rate: ≤1 request/second per domain (respects Crawl-delay)
Max depth: 3 links from seed pages
Content types: text/html only
Max page size: 2MB
Robots compliance: Full (User-agent, Disallow, Allow, Crawl-delay)
Contact: crawler@gositeme.com
Someone from somewhere
just launched website.com
Just now