During the life of a website, its quite common for pages to move around or to ‘re-structure’ your site’s files and folders. However, you can come across issues when people linking to your page go to their ‘stale’ bookmark of your site, and find that page is missing – in fact, they’ll get a ‘404 Page Not Found‘. The same will go for search engines like google and yahoo, and will adversely affect your rankings within their results.
Use mod_rewrite and .htaccess file to block user agent libwww-perl. Open your .htaccess file and add rule as follows:
Not only do I block all accesses from libwww-perl, I also log what they were looking for which turns up an amazing amount of botnet hits on a daily basis just randomly hitting websites trying to find a way inside.
You can deny acccess to a vhost just by adding the following lines in a .htaccess file:
You can enable or disable php error reportin in any vhost just by using this: php_flag display_errors on
To do this you have to create a username/password file, and a .htaccess