Why is a crawler visiting my website ?
I do not get statistical data from visiting your website.
Instead of this I use a kind of API-Meta-Crawler-System to build my word-clouds.
The crawler with the identification of "copyright sheriff" checks, if
other websites are using my statistical-content on commercial websites or if they
forgot to mention the origin of that data 8-) Have in mind, that just doing some
advertising on your website - or some Affiliate-Links is already seen as "commercial
use"!
All my usage-terms are perfectly described in the "Creative Commons
License". Have a click on the link to learn what you can do with
this content and what obligations you have to fulfill when copying content from
my website.
The copyright-sheriff checks links sporadically every 2 to 4 weeks, that
I get from my server-logs as referencing links.
How can I block this thing ?
When you want to block this crawler, it is very easy to do. First of all, turn
on mod_rewrite and copy these lines to your .htaccess file:
Options +FollowSymlinks RewriteEngine On RewriteBase / RewriteCond %{HTTP_USER_AGENT} "copyright sheriff" RewriteRule .* - [F,L]
I will have a look at that blocked websites manually.
|