Bot Traffic Is on the Rise, Prepare Your Website

For every human visiting your website, there are 3 bots.

Surprised? We were too.

But according to our internal data, we saw an average of 71.5% bot traffic since January 2025.

AI crawlers from ChatGPT (9.16%), Claude (6.09%), and other LLMs show up regularly in our bot traffic data, while search engine bots like Googlebot represent just 7.57% of traffic.

We can’t just block them all because most bots are necessary. For instance, Googlebot helps your site show up in search, or ChatGPT bot cites your brand as a source in responses.

But if you’re seeing traffic spikes or spam entries in your contact forms, that’s when you need to take action.

Just How Bad Has the Bot Traffic Surge Become?

Here is a breakdown of the bot traffic we are seeing across our customer websites on the DreamHost network:

The geographic patterns we observed are quite interesting, too.

Ireland leads bot traffic at 40.70% of daily visits, followed by Singapore at 13.04%. This geographic concentration is not just coincidence.

Ireland hosts the European headquarters of almost every major US tech company – Google, Apple, Microsoft, Amazon, and Facebook – drawn by EU access, low corporate taxes, and an English-speaking workforce.

Meanwhile, Singapore is the Asia-Pacific data center hub, with AWS, Microsoft Azure, and Google Cloud operating major facilities there to serve Asian markets with lower latency. When AI crawlers and cloud services scan websites, they’re often operating from server farms concentrated in these two strategic locations.

Our findings are quite coherent with Imperva’s Bad Bot Report 2025, where 51% of all internet traffic now comes from automated sources, with 37% classified as malicious bots.

The challenge lies in the gray area.

Some AI training crawlers (like GPTBot) collect your content to train language models without providing direct traffic benefits back to your site.

But whether these represent “good” or “bad” bots depends on how you feel about AI companies using your content for commercial purposes.

Good Bots vs. Bad Bots: What’s the Difference

Not all automated traffic deserves blocking. You need to know which bots help your business and which ones harm it before you take any action to manage bots on your website.

Good Bots That Benefit Your Site

• Search engine crawlers: For example, Googlebot and Bingbot index your content for search results.
• Social media bots: Like FacebookExternalAgent generate link previews when users share your content.
• Monitoring services: These check your site’s uptime and performance.
• AI search crawlers: Like ChatGPT-User and Perplexity-User that cite your content when answering user queries.

Bad Bots That Drain Your Resources

• Content scrapers: These steal your articles, product descriptions, and images.
• Form spam bots: They flood contact forms with junk submissions.
• Price monitoring bots: From competitors tracking your pricing strategies.
• Inventory hoarding bots: These add products to carts without purchasing.
• Credential stuffing bots: They attempt to break into user accounts.

The Hidden Costs Hitting Your Small Business Site

Irrespective of the type of bot, automated traffic impacts your website in three critical areas:

• Hosting expenses
• Security risks
• Data accuracy

Each bot visit consumes server resources just like a human visitor, but without any potential for conversions, engagement, or revenue.

1. Your Infrastructure Costs Keep Climbing

Every bot request uses your bandwidth, processing power, and storage.

If you have a content-heavy site or e-commerce platform, you are likely to see this hit your wallet fast.

The Read the Docs project reduced traffic by 75% after blocking AI crawlers and went from serving 800GB to 200GB daily, saving $1,500 per month in bandwidth costs.

If you’re on a shared hosting plan, you won’t see direct costs.

Excessive bot traffic, however, can trigger your host to throttle site performance. More aggressive bot activity can push you toward requiring a VPS or dedicated hosting solutions earlier than you planned.

2. Security and Spam Problems Get Worse

Malicious bots target your contact forms, login pages, and checkout processes.

They look for vulnerabilities, attempt credential stuffing attacks, and also flood forms with junk submissions. Since most websites have just basic security, they become an attractive target.

If your contact forms or blog comments have spam entries, I’d suggest immediately installing Askimet to see spam entries will go down. Remember, though, Askimet does not block bot traffic.

You need to take steps to block bots yourself (more on that in a bit).

3. Your Analytics Data Gets Contaminated

Bot traffic messes up your website analytics.

When bots visit pages, they don’t engage with content, creating artificially high traffic numbers with zero conversion rates.

This contamination makes it almost impossible for you to measure real user behavior, optimize conversion funnels, or make data-driven marketing decisions.

And even if Google Analytics lets you filter traffic to narrow down the analysis, it’s difficult to distinguish bot traffic from regular human visitors.

Where To Look for Bot Traffic on Your Site

There is currently no foolproof way to find all bot traffic since advanced bots are virtually indistinguishable from human visitors.

But there are certain steps you can take to start monitoring.

1. Start With DreamHost Panel’s Traffic Dashboard

If you’re hosting with DreamHost, you’re already one step ahead. The Traffic tab in your DreamHost panel gives you a clear view of traffic activity, including bot behavior. You can monitor spikes, detect anomalies, and track how many bots may be impacting your site, all without needing a third-party tool.

Our panel helps you:

• Distinguish between human and bot traffic at a glance
• View traffic by IP, referrer, or user agent
• Spot patterns early so you can take action fast

This is especially helpful for users who prefer built-in tools over complicated dashboards. Bonus – No extra configuration is needed!

2. Dig in With Google Analytics

Look at your Google Analytics dashboard for these telltale signs of heavy bot activity:

• Traffic spikes with corresponding bounce rate increases
• High page views with zero time on page
• Unusual geographic traffic patterns (sudden traffic from countries where you don’t operate)
• Referrer spam from unfamiliar domains

Go to Reports → Tech and use GA4’s tech filtering to narrow down to user technology.

Here, you can start filtering by browser and operating system, screen resolutions, device, app versions, etc. What you’re looking for are uncommon browsers, strange operating systems, and uncommon screen resolutions like:

• 1024 x 768
• 1366 x 768
• 1600 x 864
• 800 x 600
• 1600 x 1200
• 1024 x 667
• Not Specified

This would cut some real visitors from your tracking, but you’ll also see many fewer bot entries in the analytics data.

Note: Advanced bots can use normal resolutions and the latest device versions to disguise their activity. Those are extremely difficult to filter out for analysis unless you begin to screen visitors before they can view your website.

3. Check Your Server Logs

If you have access to your hosting control panel, look at your server access logs for patterns that indicate automated traffic:

• Rapid-fire requests from single IP addresses.
• Requests to non-existent pages (often bots probing for vulnerabilities).
• User agents identifying as known crawlers (both good and bad).
• Requests with no referrer information.

Many hosting providers, including DreamHost, offer log analysis tools in their control panels. Look for unusual spikes in bandwidth usage that don’t correspond with marketing campaigns or content updates.

Once you have a list of IP addresses that seem to be repeat offenders, you can start blocking the IP addresses either from your web server, like NGINX or Apache, or from a CDN.

4. Use Third-Party Monitoring Tools

Sometimes, you can only identify so much traffic. Which is why there are free, third-party tools like Cloudflare Analytics (if you use their service) which provide bot traffic breakdowns.

There are also other comprehensive apps like:

• Wordfence: For WordPress sites; identifies malicious bot attempts
• Sucuri: Offers website firewall services with bot detection
• MonsterInsights: Provides enhanced Google Analytics reporting with bot filtering

Managing and Blocking Bad Bots Without Hurting the Good Ones

I recommend going slow on blocking bot traffic because you could accidentally block real users as well. And you also need a balanced approach that blocks harmful automated traffic while preserving beneficial crawlers.

Here’s how I would implement bot management without accidentally blocking search engines or legitimate services.

Set Up Your robots.txt File

Your robots.txt file serves as the first line of defense against unwanted crawlers.

Create or update the file in your website’s root directory with specific instructions for different bot types:

# Block AI training crawlers
User-agent: GPTBot
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Google-Extended
Disallow: /

# Allow search engines
User-agent: Googlebot
Allow: /

User-agent: Bingbot
Allow: /

# Allow AI search bots that cite sources
User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

If you don’t have direct access to the robots.txt file and use WordPress, you could try using the WP Robots Txt plugin that lets you edit the file from your WordPress dashboard.

Keep in mind that robots.txt operates on an honor system. Well-behaved crawlers respect these directives, but malicious bots often ignore the file entirely.

Implement a Website Firewall

For more aggressive website security, implement a Web Application Firewall (WAF) that can block bots at the server level.

You also have a couple more options as a small business owner managing low-volume websites.

Cloudflare Free Plan:

• Provides basic bot fighting capabilities
• Blocks obvious malicious traffic automatically
• Offers customizable firewall rules for specific threats

Wordfence (WordPress Users):

• Includes bot detection and blocking features
• Real-time threat intelligence updates
• Customizable rate limiting to prevent rapid-fire requests

Server-Level Rate Limiting

Many hosting providers offer rate-limiting features that automatically slow down or block IP addresses making too many requests in short timeframes. You can configure these rate limiters to block IP addresses that browse too many pages in a very short span of time.

For instance, a normal human would require a couple of seconds to look around on a page before clicking on different links. A bot would take milliseconds to do the same.

Plugin Solutions for WordPress Sites

If you’re running WordPress, you have additional options for bot management:

• Akismet: Automatically filters comment and form spam with 99.99% accuracy using advanced AI filtering. Essential for any WordPress site accepting user submissions.
• WP Cerber Security: Provides comprehensive bot protection, including login attempt limiting, IP blocking, and advanced bot detection algorithms.
• All-In-One Security (AIOS): Offers firewall features and bot blocking capabilities suitable for small business websites.

Your Complete Bot Protection Implementation Path

I’ve split this implementation plan so it fits within tight schedules and helps you go from zero protection to a working bot defense within a short time.

20 Minutes: Quick Wins

• Update robots.txt file
• Enable basic security ​​plugins 
• Check Google Analytics for bot patterns

Start with your robots.txt file because it’s the easiest first step. Log into your website’s file manager (or use FTP) and create or update the robots.txt file in your root directory. Copy the robots.txt configuration from earlier in this article and paste it in.

If you’re running WordPress, install and activate the Wordfence security plugin right now. The free version includes basic bot protection and will start blocking obvious malicious traffic immediately.

Open Google Analytics to check your traffic sources for unusual patterns. Look for countries you don’t serve, sudden traffic spikes, or high bounce rates with zero session duration.

24 Hours: Layer Your Defense

• Set up a free Cloudflare account
• Configure rate limiting
• Add CAPTCHA to forms
• Enable comment moderation

Sign up for a free Cloudflare account and add your website. Cloudflare sits between your site and visitors, automatically blocking obvious bot traffic before it hits your server. Setup takes about 30 minutes and includes changing DNS settings, but Cloudflare walks you through each step.

Set rate limiting through your host to automatically slow down or block IP addresses making too many requests. Configure limits that allow normal browsing (a few seconds per page) but stop aggressive crawlers (multiple pages per second).

Add CAPTCHA to vulnerable forms using the reCAPTCHA plugin for WordPress or Cloudflare Turnstile. Add invisible CAPTCHA to contact forms, comment sections, and user registration pages to stop automated submissions without annoying real users.

One Week: Monitor and Optimize

• Analyze server logs
• Fine-tune firewall rules
• Set up monitoring alerts 
• Block problem IP ranges

Analyze your server access logs to identify bot patterns specific to your site. Look for rapid-fire requests from single IP addresses, requests to non-existent pages, and suspicious user agent strings. Most hosting control panels include log analysis tools that make this easier.

Use this data to create custom firewall rules in Cloudflare or your security plugin. For example, if you notice bots from specific countries consistently causing problems, you can block entire geographic regions. Or if certain user agents keep hitting your site aggressively, block those specific strings.

Set up monitoring alerts so you know when bot traffic spikes happen. Tools like UptimeRobot or Pingdom can alert you to unusual traffic patterns, slow loading times, or server resource spikes that might indicate a bot attack.

One Month: Advanced Protection

• Implement behavioral analysis
• Deploy API rate limiting
• Set up CDN caching
• Create bot honeypots

Add behavioral analysis tools like DataDome or Imperva that detect bots by analyzing mouse movements, typing patterns, and page interaction timing. These catch sophisticated bots that mimic human behavior.

Use a CDN to cache static resources aggressively. This way, bot traffic hits the cache, not your host, reducing the load from repeated requests.

One of my favorite techniques is creating bot honeypots. If you see a lot of form spam, add an extra hidden field to the form. Since bots look at the page HTML while filling forms, they will see this hidden field and add some data to it. You can safely delete all form entries that have any text in the hidden field because a human visitor cannot see it.

The Future of Bot Traffic Management

Bot traffic isn’t going away. With 71.5% of visits to our network coming from automated sources, robots now outnumber humans 3 to 1 on the internet.

But while most website owners are still figuring out what’s happening, you have everything you need to turn this bot chaos around for your business and even benefit from the good bots.

So go forth and begin implementing bot management systems while your competition is still panicking about the fake traffic!