While developing a website, you may wish to hide it from Google and other bots. This can be challenging as spectre still needs to be able to access the site without going through authorization. Here's an example of a working config:
<Location />
AuthType basic
AuthName "Dev Server"
AuthBasicProvider file
AuthUserFile /data/wre/etc/htpasswd_users
Require valid-user
# Allow access by Spectre
Order allow,deny
Allow from localhost
Satisfy any
</Location>
You may not wish to have the modperl server available for direct communication by external browsers. You can disable external access by adding the following directives to modperl.conf just before the line to Include the *modperl configurations from /data/wre/etc.
<location />
Order deny,allow
Deny from all
Allow from 127.0.0.1 67.227.162.232
</location>
Keywords: apache