# website traffic

Published articles for website traffic.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Cloudflare Adds Setting to Block AI Training Crawlers While Allowing Search Crawlers

DevFeed: [Cloudflare Adds Setting to Block AI Training Crawlers While Allowing Search Crawlers](<https://devfeed.tech/articles/cloudflare-just-gave-ai-training-bots-the-middle-finger-31387.md>)

Original publisher: [Read original article](<https://webdesignerdepot.com/cloudflare-just-gave-ai-training-bots-the-middle-finger/>)

Author: Alex Harper

Published: 2026-09-16T17:18:57Z

Content type: news

Language: en

Sources: [Web Designer Depot](<https://devfeed.tech/sources/web-designer-depot.md>)

Topics: [Cloudflare](<https://devfeed.tech/topics/cloudflare.md>), [AI Bots](<https://devfeed.tech/topics/ai-bots.md>), [Crawler](<https://devfeed.tech/topics/crawler.md>), [Google Search](<https://devfeed.tech/topics/google-search.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-crawlers](<https://devfeed.tech/tags/ai-crawlers.md>), [ai-tech](<https://devfeed.tech/tags/ai-tech.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cloudflare](<https://devfeed.tech/tags/cloudflare.md>), [content-protection](<https://devfeed.tech/tags/content-protection.md>), [future-of-the-web](<https://devfeed.tech/tags/future-of-the-web.md>), [google-extended](<https://devfeed.tech/tags/google-extended.md>), [google-search](<https://devfeed.tech/tags/google-search.md>), [googlebot](<https://devfeed.tech/tags/googlebot.md>), [openai](<https://devfeed.tech/tags/openai.md>), [publishers](<https://devfeed.tech/tags/publishers.md>), [robots-txt](<https://devfeed.tech/tags/robots-txt.md>), [search](<https://devfeed.tech/tags/search.md>), [search-engines](<https://devfeed.tech/tags/search-engines.md>), [web](<https://devfeed.tech/tags/web.md>), [web-design](<https://devfeed.tech/tags/web-design.md>), [web-development](<https://devfeed.tech/tags/web-development.md>), [web-publishing](<https://devfeed.tech/tags/web-publishing.md>), [web-scraping](<https://devfeed.tech/tags/web-scraping.md>), [website-traffic](<https://devfeed.tech/tags/website-traffic.md>)

### AI overview

Cloudflare launched a Disallow AI Training setting that lets website owners allow traditional search crawlers while blocking training-only crawlers from companies including Amazon, Anthropic, Meta, and OpenAI. The article notes that robots.txt depends on crawler compliance and that blocking Google-Extended does not remove content from Google Search features such as AI Overviews or AI Mode.

### Source excerpt

Cloudflare just gave website owners a new weapon against AI crawlers: keep the search traffic, block the AI training. After years of watching bots consume the web's content, publishers finally have an easier way to tell AI companies where to go.

## How Changing a Blog's Domain Affected SEO Performance and Traffic

DevFeed: [How Changing a Blog's Domain Affected SEO Performance and Traffic](<https://devfeed.tech/articles/seo-kicks-my-stats-27705.md>)

Original publisher: [Read original article](<https://gagor.pro/2024/03/seo-kicks-my-stats/>)

Author: Tom

Published: 2024-03-03T00:00:00Z

Content type: article

Language: en

Sources: [Tomasz Gągor](<https://devfeed.tech/sources/tomasz-gagor.md>)

Topics: [Search engine optimization (SEO)](<https://devfeed.tech/topics/seo.md>), [Website](<https://devfeed.tech/topics/website.md>), [data](<https://devfeed.tech/topics/data.md>)

Tags: [domain](<https://devfeed.tech/tags/domain.md>), [domain-change-seo](<https://devfeed.tech/tags/domain-change-seo.md>), [domain-rank-analysis](<https://devfeed.tech/tags/domain-rank-analysis.md>), [global](<https://devfeed.tech/tags/global.md>), [international-seo](<https://devfeed.tech/tags/international-seo.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [multilingual-content](<https://devfeed.tech/tags/multilingual-content.md>), [performance](<https://devfeed.tech/tags/performance.md>), [seo](<https://devfeed.tech/tags/seo.md>), [seo-impact](<https://devfeed.tech/tags/seo-impact.md>), [strategy](<https://devfeed.tech/tags/strategy.md>), [website-traffic](<https://devfeed.tech/tags/website-traffic.md>)

### AI overview

The author describes moving a bilingual blog from timor.site to gagor.pl after a domain renewal became more expensive. Despite redirecting the content without changes, search impressions and traffic differed between the domains, and the author plans to organize the site by language to improve international visibility and SEO alignment.

### Source excerpt

Explore the impact of domain changes on SEO performance and traffic, and learn strategies to improve international visibility with multilingual content.